菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-07-08
📄 Abstract - General Incomplete Multimodal Learning via Dynamic Quality Perception

Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: this https URL.

顶级标签: multi-modal machine learning
详细标签: missing modalities quality perception adaptive fusion noise estimation robust learning 或 搜索:

基于动态质量感知的通用不完整多模态学习 / General Incomplete Multimodal Learning via Dynamic Quality Perception


1️⃣ 一句话总结

本文提出一种名为GIML的统一框架,通过动态感知各模态的噪声与退化程度,同时处理模态完全缺失和模态内容严重受损这两种不完整情况,从而提升多模态模型在实际应用中的鲁棒性和泛化能力。

源自 arXiv: 2607.06943