从固定相机到自由相机:无需标定的视角鲁棒视觉语言动作模型 / From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
1️⃣ 一句话总结
本文提出了一种名为CamVLA的新型视觉语言动作模型,它通过让机器人自主从相机视角中推导空间关系,而非依赖外部标定信息,从而在相机位置改变时仍能稳定执行任务,仅需一张普通彩色图片即可工作。
Real-world robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust Vision-Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye matrix relating cameras to the robot base. A deterministic geometric transformation composes the two predictions into a robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: this https URL.
从固定相机到自由相机:无需标定的视角鲁棒视觉语言动作模型 / From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
本文提出了一种名为CamVLA的新型视觉语言动作模型,它通过让机器人自主从相机视角中推导空间关系,而非依赖外部标定信息,从而在相机位置改变时仍能稳定执行任务,仅需一张普通彩色图片即可工作。
源自 arXiv: 2607.05396