迈向有效的多模态图基础模型:一种基于分治策略的方法 / Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach
1️⃣ 一句话总结
这篇论文提出了一种名为PLANET的新框架,它通过分治策略解决了现有多模态图基础模型在模态交互与对齐上的不足,从而在多种图相关和多模态生成任务上取得了更优的性能。
Graph Foundation Models (GFMs) have achieved remarkable success in generalizing across diverse domains. However, they mainly focus on Text-Attributed Graphs (TAGs), leaving Multimodal-Attributed Graphs (MAGs) largely untapped. Developing Multimodal Graph Foundation Models (MGFMs) allows for leveraging the rich multimodal information in MAGs, and extends applicability to broader types of downstream tasks. While recent MGFMs integrate diverse modality information, our empirical investigation reveals two fundamental limitations of existing MGFMs: (1)they fail to explicitly model modality interaction, essential for capturing intricate cross-modal semantics beyond simple aggregation, and (2)they exhibit sub-optimal modality alignment, which is critical for bridging the significant semantic disparity between distinct modal spaces. To address these challenges, we propose PLANET (graPh topoLogy-aware modAlity iNteraction and alignmEnT), a novel framework employing a Divide-and-Conquer strategy to decouple modality interaction and alignment across distinct granularities. At the embedding granularity, (1)Embedding-wise Domain Gating (EDG) performs local semantic enrichment by adaptively infusing topology-aware cross-modal context, achieving modality interaction. At the node granularity, (2)Node-wise Discretization Retrieval (NDR) ensures global modality alignment by constructing a Discretized Semantic Representation Space (DSRS) to bridge modality gaps. Extensive experiments demonstrate that PLANET significantly outperforms state-of-the-art baselines across diverse graph-centric and multimodal generative tasks.
迈向有效的多模态图基础模型:一种基于分治策略的方法 / Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach
这篇论文提出了一种名为PLANET的新框架,它通过分治策略解决了现有多模态图基础模型在模态交互与对齐上的不足,从而在多种图相关和多模态生成任务上取得了更优的性能。
源自 arXiv: 2602.04116