菜单

关于 🐙 GitHub
arXiv 提交日期: 2026-06-10
📄 Abstract - Q-Fold: Query-Aware Focus-Context Spatio-Temporal Folding for Long Video Understanding

Long-video understanding remains challenging for multimodal large language models, because temporally extended videos often contain thousands of frames and are therefore expensive to process exhaustively. Existing methods usually construct compact visual inputs from long videos under a limited visual budget. However, most of them still follow a frame-centric paradigm and apply similar representations to retained content regardless of its importance. This makes it difficult to preserve both high-fidelity visual evidence and broad temporal coverage. To address this issue, we propose Q-Fold, a training-free input construction framework for long-video understanding. Instead of treating isolated frames as the basic modeling unit, Q-Fold operates on contiguous temporal segments and constructs a heterogeneous Focus--Context representation under query guidance. Query-relevant segments are preserved as high-fidelity Focus Frames, while less relevant segments are folded into chronology-preserving contextual layouts. In this way, Q-Fold preserves critical visual evidence and broad temporal coverage, while better maintaining local temporal continuity within short segments. Experiments on four long-video benchmarks with multiple Video-MLLMs show that Q-Fold consistently improves performance without increasing the input budget. Notably, it achieves gains of up to 9.1 percentage points on an ultra-long video benchmark. Code will be made publicly available.

顶级标签: multi-modal video model evaluation
详细标签: long video understanding multimodal large language models input construction focus-context temporal segments 或 搜索:

Q-Fold:面向长视频理解的查询感知式焦点-上下文时空折叠方法 / Q-Fold: Query-Aware Focus-Context Spatio-Temporal Folding for Long Video Understanding


1️⃣ 一句话总结

本文提出了一种无需额外训练的输入构建框架Q-Fold,通过根据用户查询将长视频中相关片段保留为高保真帧、不相关片段折叠成保持时间顺序的紧凑布局,从而在有限计算资源下同时保留关键视觉证据和广泛时间覆盖,显著提升了多模态大模型在长视频理解任务上的表现。

源自 arXiv: 2606.12125