SMSP:一种用于多模态大语言模型感知视觉错觉的多尺度感知即插即用策略 / SMSP: A Plug-and-Play Strategy of Multi-Scale Perception for MLLMs to Perceive Visual Illusions
1️⃣ 一句话总结
本文发现多模态大语言模型容易受视觉错觉图像中的高频背景干扰而忽略隐藏内容,并提出了一种即插即用的多尺度感知策略来抑制干扰、提升模型感知能力,使其更接近人类视觉。
Recent works have shown that Multimodal Large Language Models (MLLMs) are highly vulnerable to hidden-pattern visual illusions, where the hidden content is imperceptible to models but obvious to humans. This deficiency highlights a perceptual misalignment between current MLLMs and humans, and also introduces potential safety concerns. To systematically investigate this failure, we introduce IlluChar, a comprehensive and challenging illusion dataset, and uncover a key underlying mechanism for the models' failure: high-frequency attention bias, where the models are easily distracted by high-frequency background textures in illusion images, causing them to overlook hidden patterns. To address the issue, we propose the Strategy of Multi-Scale Perception (SMSP), a plug-and-play framework that aligns with human visual perceptual strategies. By suppressing distracting high-frequency backgrounds, SMSP generates images closer to human perception. Our experiments demonstrate that SMSP significantly improves the performance of all evaluated MLLMs on illusion images, for instance, increasing the accuracy of Qwen3-VL-8B-Instruct from 13.0% to 84.0%. Our work provides novel insights into MLLMs' visual perception, and offers a practical and robust solution to enhance it. Our code is publicly available at this https URL.
SMSP:一种用于多模态大语言模型感知视觉错觉的多尺度感知即插即用策略 / SMSP: A Plug-and-Play Strategy of Multi-Scale Perception for MLLMs to Perceive Visual Illusions
本文发现多模态大语言模型容易受视觉错觉图像中的高频背景干扰而忽略隐藏内容,并提出了一种即插即用的多尺度感知策略来抑制干扰、提升模型感知能力,使其更接近人类视觉。
源自 arXiv: 2603.23118