菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-05-27
📄 Abstract - ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning

Multimodal Large Language Models (MLLMs) have increasingly localized and interleaved visual evidence for deliberative reasoning. Grounding-based approaches typically focus on regions of interest (RoIs) by injecting cropped image patches or RoI-specific features into the reasoning context. However, such designs can weaken holistic scene understanding and inter-object relations, while incurring decoding costs that scale with the number and size of RoIs. Alternatively, adaptive visual feature selection often requires fine-grained supervision or complex heuristics. To address these limitations, we propose ROVER (Routing Object-centric Visual Evidence for grounded multi-image Reasoning), a lightweight, learnable plugin for efficient global visual evidence routing. Upon each object grounding prediction, ROVER injects a step-specific token triplet to synergistically: (i) aggregate the ongoing reasoning context, (ii) distill intra-image cues into a visual working space via object-centric differential attention, and (iii) route and integrate history-aware evidence across objects and images within this space for subsequent reasoning. We integrate ROVER into Qwen2.5-VL-7B and develop an interleaved SFT-to-GRPO training pipeline. Strictly adhering to the original datasets and evaluation protocols, our method achieves the best performance on MM-GCoT (+4.8% answer accuracy, +14.6% grounding accuracy) and VideoEspresso (+8.6% answer accuracy). The VideoEspresso-trained model demonstrates strong transferability, outperforming the base model by +4.7% on average across diverse benchmarks.

顶级标签: multi-modal model training model evaluation
详细标签: multimodal llm visual grounding object-centric reasoning multi-image reasoning routing 或 搜索:

ROVER:面向对象中心的视觉证据路由实现多图像推理 / ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning


1️⃣ 一句话总结

ROVER是一个轻量级插件,通过为每个检测到的物体插入特殊的“令牌三元组”,高效地将多张图像中的视觉证据结合到推理过程中,从而在不牺牲全局场景理解的前提下,显著提升多图像问答任务中的答案准确性和定位准确性。

源自 arXiv: 2605.27959