菜单

关于 🐙 GitHub
arXiv 提交日期: 2025-12-19
📄 Abstract - SAM Audio: Segment Anything in Audio

General audio source separation is a key capability for multimodal AI systems that can perceive and reason about sound. Despite substantial progress in recent years, existing separation models are either domain-specific, designed for fixed categories such as speech or music, or limited in controllability, supporting only a single prompting modality such as text. In this work, we present SAM Audio, a foundation model for general audio separation that unifies text, visual, and temporal span prompting within a single framework. Built on a diffusion transformer architecture, SAM Audio is trained with flow matching on large-scale audio data spanning speech, music, and general sounds, and can flexibly separate target sources described by language, visual masks, or temporal spans. The model achieves state-of-the-art performance across a diverse suite of benchmarks, including general sound, speech, music, and musical instrument separation in both in-the-wild and professionally produced audios, substantially outperforming prior general-purpose and specialized systems. Furthermore, we introduce a new real-world separation benchmark with human-labeled multimodal prompts and a reference-free evaluation model that correlates strongly with human judgment.

顶级标签: audio multi-modal model training
详细标签: audio source separation diffusion transformer flow matching multimodal prompting foundation model 或 搜索:

SAM音频:分割任意音频 / SAM Audio: Segment Anything in Audio


1️⃣ 一句话总结

这篇论文提出了一个名为SAM Audio的通用音频分割基础模型,它能够通过文本、视觉或时间片段等多种提示方式,灵活地从混合音频中分离出目标声音(如语音、音乐或一般声响),并在多个基准测试中取得了领先性能。

源自 arXiv: 2512.18099