BEAT:用于智能音乐引导电影预告片生成的节奏弹性对齐框架 / BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation
1️⃣ 一句话总结
本文提出BEAT框架,通过引入弹性节奏对齐技术,让电影预告片自动剪辑时能像专业剪辑师一样根据音乐强弱灵活调整镜头切换节奏,从而生成更自然、高质量的音乐与画面同步的预告片。
Automatic movie trailer generation must select shots from a full-length film and synchronize them with background music. Existing methods either relegate music alignment to post-processing or enforce rigid one-to-one shot-music mappings, overlooking that professional editing rhythm is elastic: rapid cuts accompany high-energy passages while sustained shots span quieter bars. We introduce BEAT, a framework that addresses this gap with two core components: MuVA, a compact music-visual alignment encoder trained with Sinkhorn-regularized two-stage learning, and Bar-DP, an energy-adaptive dynamic programming algorithm that produces elastic many-to-one alignments following musical dynamics. These components are integrated into a five-phase agentic pipeline that grounds the core alignment in learned cross-modal features while coordinating higher-level creative decisions through structured text signals. To support comprehensive evaluation, we also introduce TrailerArena, a benchmark with 20+ metrics across four complementary dimensions. On TrailerArena, BEAT achieves state-of-the-art performance across shot selection, ordering, and perceptual quality, while producing fully composed trailers end-to-end.
BEAT:用于智能音乐引导电影预告片生成的节奏弹性对齐框架 / BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation
本文提出BEAT框架,通过引入弹性节奏对齐技术,让电影预告片自动剪辑时能像专业剪辑师一样根据音乐强弱灵活调整镜头切换节奏,从而生成更自然、高质量的音乐与画面同步的预告片。
源自 arXiv: 2605.27067