4 papers
UniAE-MoE: A Unified Audio Encoder via Mixture of Experts
Shengbo Cai, Zhisheng Zhang, Zichao Nie +3
Large Audio Language Models (LALMs) rely on effective audio encoders for multi-task performance. We introduce UniAE-MoE, a unified audio encoder designed to model cross-domain audi…
Discrete vs. Continuous: A Comprehensive Study of Unified Audio Understanding in LALMs
Jing Peng, Zichao Nie, Zhisheng Zhang +2
Large Audio Language Models (LALMs) utilize either continuous features or discrete tokens, yet the optimal representation paradigm for general audio understanding remains debated.…
LoSATok: Low-Dimensional Semantic-Acoustic Tokenizer for Cross-Domain Audio Understanding and Generation
Zhisheng Zhang, Xiang Li, Yixuan Zhou +3
Audio tokenizers are fundamental to unifying audio understanding and generation. Understanding requires high-level semantics, while generation demands semantic and acoustic details…
ParaLBench: A Large-Scale Benchmark for Computational Paralinguistics over Acoustic Foundation Models
Zixing Zhang, Weixiang Xu, Zhongren Dong +5
Computational paralinguistics (ComParal) aims to develop algorithms and models to automatically detect, analyze, and interpret non-verbal information from speech communication, e.…