菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-07-08
📄 Abstract - GeoProp: Grounding Robot State in Vision for Generalist Manipulation

Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit alignment with visual tokens. Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the robot's state within the scene, frequently underperforming even vision-only baselines. To address this, we introduce GeoProp, a lightweight, plug-and-play adapter that aligns proprioception with vision through explicit geometric grounding and spatial feature sampling. GeoProp projects the robot state onto the image plane to sample localized visual features, constructing a grounded state token. It then injects state-derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short-horizon predicted coordinate derived from recent kinematics, providing look-ahead visual context. Across 67 tasks, GeoProp improves Diffusion Policy by 8.7% on 63 simulation tasks and pi_0 by 4.0% on the RoboTwin subset, and yields a 10.6% average gain across both policy families in the real world, while adding only 2-3% to the parameter count. These results demonstrate that GeoProp is a simple yet high-impact inductive bias for generalist embodied policies. Project page: this https URL.

顶级标签: robotics
详细标签: proprioception vision alignment state grounding manipulation diffusion policy 或 搜索:

GeoProp:通过视觉锚定机器人状态实现通用操作 / GeoProp: Grounding Robot State in Vision for Generalist Manipulation


1️⃣ 一句话总结

本文提出了一种轻量级即插即用模块GeoProp,通过将机器人关节角度等本体感觉信息投影到图像平面上,并从中采样空间特征,让机器人能像人类一样“看到”自己手臂的位置和运动意图,从而使操作策略在模拟和真实任务中平均提升4%至10%的性能,且几乎不增加模型计算量。

源自 arXiv: 2607.07101