超越捷径:通过定性推理缓解冻结视觉语言模型中的视觉错觉 / Beyond Shortcuts: Mitigating Visual Illusions in Frozen VLMs via Qualitative Reasoning
1️⃣ 一句话总结
本文提出了一种无需训练的数据驱动框架SQI,通过注入公理约束、分解场景和反事实自我验证三个步骤,让冻结的视觉语言模型在遇到光学错觉图片时,能依靠定性推理而非语言捷径,更准确地理解图像,从而在不调整模型参数的前提下显著提升抗错觉能力。
While Vision-Language Models (VLMs) have achieved state-of-the-art performance in general visual tasks, their perceptual robustness remains remarkably brittle when confronted with optical illusions. These failures are often attributed to shortcut heuristics, where models prioritize linguistic priors and memorized prototypes over direct visual evidence. In this work, we propose Structured Qualitative Inference (SQI), a training-free, data-centric framework designed to fortify visual grounding in frozen VLMs. SQI addresses perceptual anomalies through three systematic modules: (1) Axiomatic Constraint Injection, which suppresses erroneous metric estimations and quantitative hallucinations; (2) Hierarchical Scene Decomposition, which decouples target visual manifolds from complex background distractors; and (3) Counterfactual Self-Verification, an adversarial reasoning step that mitigates confirmation bias. By orchestrating these qualitative constraints at inference time, SQI effectively aligns high-level linguistic reasoning with low-level visual perception. Our framework was evaluated on the DataCV 2026 Challenge (Task I: Classic Illusion Understanding), where it ranked 2nd place overall. Experimental results demonstrate that SQI not only significantly enhances accuracy across diverse illusion categories but also provides superior diagnostic interpretability without any model fine-tuning. Our success underscores the potential of structured qualitative grounding as a robust paradigm for developing next-generation, illusion-resistant vision-language systems.
超越捷径:通过定性推理缓解冻结视觉语言模型中的视觉错觉 / Beyond Shortcuts: Mitigating Visual Illusions in Frozen VLMs via Qualitative Reasoning
本文提出了一种无需训练的数据驱动框架SQI,通过注入公理约束、分解场景和反事实自我验证三个步骤,让冻结的视觉语言模型在遇到光学错觉图片时,能依靠定性推理而非语言捷径,更准确地理解图像,从而在不调整模型参数的前提下显著提升抗错觉能力。
源自 arXiv: 2604.26250