GeoProp:通过视觉锚定机器人状态实现通用操作 / GeoProp: Grounding Robot State in Vision for Generalist Manipulation
1️⃣ 一句话总结
本文提出了一种轻量级即插即用模块GeoProp,通过将机器人关节角度等本体感觉信息投影到图像平面上,并从中采样空间特征,让机器人能像人类一样“看到”自己手臂的位置和运动意图,从而使操作策略在模拟和真实任务中平均提升4%至10%的性能,且几乎不增加模型计算量。
Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit alignment with visual tokens. Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the robot's state within the scene, frequently underperforming even vision-only baselines. To address this, we introduce GeoProp, a lightweight, plug-and-play adapter that aligns proprioception with vision through explicit geometric grounding and spatial feature sampling. GeoProp projects the robot state onto the image plane to sample localized visual features, constructing a grounded state token. It then injects state-derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short-horizon predicted coordinate derived from recent kinematics, providing look-ahead visual context. Across 67 tasks, GeoProp improves Diffusion Policy by 8.7% on 63 simulation tasks and pi_0 by 4.0% on the RoboTwin subset, and yields a 10.6% average gain across both policy families in the real world, while adding only 2-3% to the parameter count. These results demonstrate that GeoProp is a simple yet high-impact inductive bias for generalist embodied policies. Project page: this https URL.
GeoProp:通过视觉锚定机器人状态实现通用操作 / GeoProp: Grounding Robot State in Vision for Generalist Manipulation
本文提出了一种轻量级即插即用模块GeoProp,通过将机器人关节角度等本体感觉信息投影到图像平面上,并从中采样空间特征,让机器人能像人类一样“看到”自己手臂的位置和运动意图,从而使操作策略在模拟和真实任务中平均提升4%至10%的性能,且几乎不增加模型计算量。
源自 arXiv: 2607.07101