ReCALL:针对基于MLLM的组合图像检索的能力退化再校准 / ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval
1️⃣ 一句话总结
这篇论文提出了一个名为ReCALL的通用框架,通过诊断、生成和精炼三个步骤,解决了将生成式多模态大语言模型直接用于图像检索任务时,其原有的精细推理能力会下降的问题,从而显著提升了组合图像检索的性能。
Composed Image Retrieval (CIR) aims to retrieve target images based on a hybrid query comprising a reference image and a modification text. Early dual-tower Vision-Language Models (VLMs) struggle with cross-modality compositional reasoning required for this task. Recently, adapting generative Multimodal Large Language Models (MLLMs) for retrieval offers a promising direction. However, we identify that this adaptation strategy overlooks a fundamental issue: adapting a generative MLLM into a single-embedding discriminative retriever triggers a paradigm conflict, which leads to Capability Degradation - the deterioration of native fine-grained reasoning after retrieval adaptation. To address this challenge, we propose ReCALL (Recalibrating Capability Degradation), a model-agnostic framework that follows a diagnose-generate-refine pipeline: Firstly, we diagnose cognitive blind spots of the retriever via self-guided informative instance mining. Next, we generate corrective instructions and triplets by CoT prompting the foundation MLLM and conduct quality control with VQA-based consistency filtering. Finally, we refine the retriever through continual training on these triplets with a grouped contrastive scheme, thereby internalizing fine-grained visual-semantic distinctions and realigning the discriminative embedding space of retriever with intrinsic compositional reasoning within the MLLM. Extensive experiments on CIRR and FashionIQ show that ReCALL consistently recalibrates degraded capabilities and achieves state-of-the-art performance. Code will be released soon.
ReCALL:针对基于MLLM的组合图像检索的能力退化再校准 / ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval
这篇论文提出了一个名为ReCALL的通用框架,通过诊断、生成和精炼三个步骤,解决了将生成式多模态大语言模型直接用于图像检索任务时,其原有的精细推理能力会下降的问题,从而显著提升了组合图像检索的性能。
源自 arXiv: 2602.01639