超越视觉检测:合成医学图像检测的多模态鲁棒性审计 / Beyond Visual Forensics: Auditing Multimodal Robustness for Synthetic Medical Image Detection
1️⃣ 一句话总结
该论文发现,在临床多模态场景下,视觉语言模型判断医学图像真伪时,会因附带的文本记录不同而对同一张图像给出不同结论;为此,作者设计了一套配对基准测试,用于系统评估和提升此类模型在多模态输入下的鲁棒性。
With the rapid adoption of generative AI, synthetic medical images pose growing risks, including diagnostic deception and insurance fraud. Although prior work has explored vision-language model (VLM)-based synthetic image detection, these evaluations typically consider images in isolation. In clinical practice, however, images are interpreted alongside structured records and metadata, and VLMs are increasingly deployed under joint image-record inputs. We uncover a previously underexamined multimodal vulnerability: when given both modalities, VLMs may overweight record context in authenticity judgments, such that the same image receives different predictions solely due to changes in its accompanying text. This raises concerns about robustness in real-world deployment. To systematically characterize this effect, we reformulate synthetic medical image detection as an audit of multimodal robustness at the image-record interface and introduce a paired benchmark that holds the image fixed while swapping controlled metadata variants. Across multiple imaging modalities, we evaluate diverse open-weight and frontier API VLMs and quantify how metadata alone shifts authenticity predictions. Our benchmark provides a standardized tool for assessing and improving multimodal robustness beyond image-only settings. The code is available at this https URL.
超越视觉检测:合成医学图像检测的多模态鲁棒性审计 / Beyond Visual Forensics: Auditing Multimodal Robustness for Synthetic Medical Image Detection
该论文发现,在临床多模态场景下,视觉语言模型判断医学图像真伪时,会因附带的文本记录不同而对同一张图像给出不同结论;为此,作者设计了一套配对基准测试,用于系统评估和提升此类模型在多模态输入下的鲁棒性。
源自 arXiv: 2606.25375