菜单

关于 🐙 GitHub
arXiv 提交日期: 2025-12-14
📄 Abstract - Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space

Recent advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced cross-modal understanding and reasoning by incorporating Chain-of-Thought (CoT) reasoning in the semantic space. Building upon this, recent studies extend the CoT mechanism to the visual modality, enabling models to integrate visual information during reasoning through external tools or explicit image generation. However, these methods remain dependent on explicit step-by-step reasoning, unstable perception-reasoning interaction and notable computational overhead. Inspired by human cognition, we posit that thinking unfolds not linearly but through the dynamic interleaving of reasoning and perception within the mind. Motivated by this perspective, we propose DMLR, a test-time Dynamic Multimodal Latent Reasoning framework that employs confidence-guided latent policy gradient optimization to refine latent think tokens for in-depth reasoning. Furthermore, a Dynamic Visual Injection Strategy is introduced, which retrieves the most relevant visual features at each latent think token and updates the set of best visual patches. The updated patches are then injected into latent think token to achieve dynamic visual-textual interleaving. Experiments across seven multimodal reasoning benchmarks and various model architectures demonstrate that DMLR significantly improves reasoning and perception performance while maintaining high inference efficiency.

顶级标签: multi-modal llm model evaluation
详细标签: multimodal reasoning latent space dynamic interleaving chain-of-thought inference efficiency 或 搜索:

思维内推理:潜在空间中的动态多模态交错 / Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space


1️⃣ 一句话总结

这篇论文提出了一种名为DMLR的新方法,它模仿人类思维中感知与推理动态交织的过程,在模型内部潜在空间中进行高效的视觉-文本信息融合,从而显著提升了多模态模型的推理能力和效率。


源自 arXiv: 2512.12623