Think3D:利用空间进行空间推理的思考框架 / Think3D: Thinking with Space for Spatial Reasoning
1️⃣ 一句话总结
这篇论文提出了一个名为Think3D的框架,它能让现有的视觉大模型通过操控三维重建场景来‘思考’空间关系,无需额外训练即可显著提升它们在三维空间推理任务上的表现。
Understanding and reasoning about the physical world requires spatial intelligence: the ability to interpret geometry, perspective, and spatial relations beyond 2D perception. While recent vision large models (VLMs) excel at visual understanding, they remain fundamentally 2D perceivers and struggle with genuine 3D reasoning. We introduce Think3D, a framework that enables VLM agents to think with 3D space. By leveraging 3D reconstruction models that recover point clouds and camera poses from images or videos, Think3D allows the agent to actively manipulate space through camera-based operations and ego/global-view switching, transforming spatial reasoning into an interactive 3D chain-of-thought process. Without additional training, Think3D significantly improves the spatial reasoning performance of advanced models such as GPT-4.1 and Gemini 2.5 Pro, yielding average gains of +7.8% on BLINK Multi-view and MindCube, and +4.7% on VSI-Bench. We further show that smaller models, which struggle with spatial exploration, benefit significantly from a reinforcement learning policy that enables the model to select informative viewpoints and operations. With RL, the benefit from tool usage increases from +0.7% to +6.8%. Our findings demonstrate that training-free, tool-augmented spatial exploration is a viable path toward more flexible and human-like 3D reasoning in multimodal agents, establishing a new dimension of multimodal intelligence. Code and weights are released at this https URL.
Think3D:利用空间进行空间推理的思考框架 / Think3D: Thinking with Space for Spatial Reasoning
这篇论文提出了一个名为Think3D的框架,它能让现有的视觉大模型通过操控三维重建场景来‘思考’空间关系,无需额外训练即可显著提升它们在三维空间推理任务上的表现。
源自 arXiv: 2601.13029