📄 论文总结
G²VLM:基于几何基础的视觉语言模型,统一3D重建与空间推理 / G$^2$VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning
1️⃣ 一句话总结
这篇论文提出了一个名为G²VLM的视觉语言模型,它通过整合3D重建和空间理解能力,显著提升了AI在空间推理任务上的表现,无需依赖大量难以获取的3D标注数据即可实现高效学习。
Vision-Language Models (VLMs) still lack robustness in spatial intelligence, demonstrating poor performance on spatial understanding and reasoning tasks. We attribute this gap to the absence of a visual geometry learning process capable of reconstructing 3D space from 2D images. We present G$^2$VLM, a geometry grounded vision-language model that bridges two fundamental aspects of spatial intelligence: spatial 3D reconstruction and spatial understanding. G$^2$VLM natively leverages learned 3D visual geometry features to directly predict 3D attributes and enhance spatial reasoning tasks via in-context learning and interleaved reasoning. Our unified design is highly scalable for spatial understanding: it trains on abundant multi-view image and video data, while simultaneously leveraging the benefits of 3D visual priors that are typically only derived from hard-to-collect annotations. Experimental results demonstrate G$^2$VLM is proficient in both tasks, achieving comparable results to state-of-the-art feed-forward 3D reconstruction models and achieving better or competitive results across spatial understanding and reasoning tasks. By unifying a semantically strong VLM with low-level 3D vision tasks, we hope G$^2$VLM can serve as a strong baseline for the community and unlock more future applications, such as 3D scene editing.
G²VLM:基于几何基础的视觉语言模型,统一3D重建与空间推理 / G$^2$VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning
这篇论文提出了一个名为G²VLM的视觉语言模型,它通过整合3D重建和空间理解能力,显著提升了AI在空间推理任务上的表现,无需依赖大量难以获取的3D标注数据即可实现高效学习。