菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-03-04
📄 Abstract - Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual Understanding

Fine-grained visual understanding is shifting from static classification to knowledge-augmented reasoning, where models must justify as well as recognise. Existing approaches remain limited by closed-set taxonomies and single-label prediction, leading to significant degradation under open-set or context-dependent conditions. We present the Knowledge-Augmented Fine-Grained Reasoning Agent (KFRA), a unified framework that transforms fine-grained perception into evidence-driven reasoning. KFRA operates through a three-stage closed reasoning loop that emulates expert analysis. It first performs open-vocabulary detection and web-scale retrieval to generate category hypotheses. It then conducts discriminative regions localisation by aligning textual knowledge with visual evidence through a global-to-local focusing mechanism. Finally, it integrates all multimodal evidence within a large multimodal model to perform interpretable reasoning. Unlike existing agents that treat retrieval and reasoning as independent processes, KFRA establishes a retrieval-grounding coupling that converts retrieved knowledge into spatially grounded evidence for verification. This design enables factual, interpretable, and task-agnostic reasoning across diverse fine-grained scenarios. To evaluate this capability, we construct FGExpertBench, a benchmark designed to assess reasoning depth and cross-task generalisation across six knowledge dimensions. Extensive experiments demonstrate that KFRA consistently surpasses both standalone large multimodal models and current agent frameworks, achieving up to 19 percent improvement in reasoning accuracy and delivering evidence-grounded interpretability in open-set fine-grained visual understanding.

顶级标签: computer vision multi-modal agents
详细标签: fine-grained visual understanding knowledge-augmented reasoning open-set recognition evidence-driven reasoning multimodal agent 或 搜索:

像专家一样观察:一个用于开放集细粒度视觉理解的知识增强智能体 / Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual Understanding


1️⃣ 一句话总结

这篇论文提出了一个名为KFRA的知识增强智能体,它通过模仿专家的三步分析过程(提出假设、定位关键区域、整合多模态证据进行推理),将细粒度视觉识别转化为基于证据的推理,从而在开放环境下更准确、更可解释地理解复杂图像。

源自 arXiv: 2603.03762