Audio-text retrieval systems based on Contrastive Language-Audio Pretraining (CLAP) achieve strong performance on traditional benchmarks; however, these benchmarks rely on caption-style queries that differ substantially from real-world search behavior, limiting their assessment of practical retrieval robustness.
Omni-Embed-Audio (OEA)
We present Omni-Embed-Audio (OEA), a retrieval-oriented encoder leveraging multimodal LLMs with native audio understanding.
User-Intent Queries (UIQs)
To systematically evaluate robustness beyond caption-style queries, we introduce User-Intent Queries (UIQs) - five formulations reflecting natural search behaviors: questions, commands, keyword tags, paraphrases, and exclusion-based negative queries.
Hard Negative Mining Pipeline and Discrimination Metrics
For negative queries, we develop a hard negative mining pipeline and propose discrimination metrics (HNSR, TFR) assessing models' ability to suppress acoustically similar distractors.
Experiments and Results
Experiments on AudioCaps, Clotho, and MECAT show that OEA achieves comparable text-to-audio retrieval performance to state-of-the-art M2D-CLAP, while demonstrating clear advantages in two critical areas:
- (1) dominant text-to-text retrieval (+22% relative improvement),
- (2) substantially superior hard negative discrimination (+4.3%p HNSR@10, +34.7% relative TFR@10),
revealing that LLM backbones provide superior semantic understanding of complex queries.