SelectTSL:复杂场景中基于提示引导的选择性目标声音定位 / SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarios
1️⃣ 一句话总结
本文提出了一种名为SelectTSL的深度学习模型,能够根据用户提供的提示(如音频描述),在多个声音交错的复杂环境中,只定位用户指定的特定声源的方向和数量,而忽略其他无关声音,解决了现有技术无法同时实现选择性定位和保持空间精度的难题。
Humans can selectively attend to a target sound and estimate its direction in complex scenarios, whereas such selective localization remains challenging for current deep learning-based systems. Sound source localization (SSL) has achieved remarkable success with deep learning, yet most methods localize all active sources without selectivity. Conversely, target sound extraction (TSE) extracts sources using multimodal prompts but typically fails to preserve the multichannel spatial information required for accurate localization. To bridge this gap, we formulate the task of prompt-guided selective target sound localization and propose SelectTSL, an end-to-end architecture that localizes only the user-specified target in multi-source acoustic scenes. Specifically, we design a target-aware selective localization strategy that employs a Prompt-Guided Selective Attention Module (PGSA) to generate prompt-informed embeddings. These embeddings guide an inter-channel phase difference (IPD) enhancer to refine raw phase cues, fusing with target magnitudes to jointly estimate direction of arrival (DoA) and target-source cardinality, i.e., the number of target sound sources. This coupled design effectively focuses on the user-specified target spatial cues for selective localization and also handles time-varying numbers of target sources. Extensive experiments on both synthetic data and real-world recordings demonstrate that our proposed method consistently outperforms other baselines and exhibits robust generalization to real acoustic environments.
SelectTSL:复杂场景中基于提示引导的选择性目标声音定位 / SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarios
本文提出了一种名为SelectTSL的深度学习模型,能够根据用户提供的提示(如音频描述),在多个声音交错的复杂环境中,只定位用户指定的特定声源的方向和数量,而忽略其他无关声音,解决了现有技术无法同时实现选择性定位和保持空间精度的难题。
源自 arXiv: 2607.02343