23 papers
Imitation Learning for Elder-Facing Speech Synthesis
Dongrui Han, Weidong Chen, Jiawen Kang +3
Recent advances in text-to-speech (TTS) synthesis have achieved highly natural and expressive speech generation. However, these systems are designed for general adults and overlook…
Lamer-SSL: Layer-aware Mixture of LoRA Experts for Continual Multilingual Expansion of Self-supervised Models without Forgetting
Jing Xu, Minglin Wu, Xueyuan Chen +2
Despite their impressive performance, self-supervised speech models often struggle to generalize to new languages and tend to forget previously acquired knowledge during continual…
UniAudio 2.0: A Unified Audio Language Model with Text-Aligned Factorized Audio Tokenization
Dongchao Yang, Yuanyuan Wang, Dading Chong +3
We study two foundational problems in audio language models: (1) how to design an audio tokenizer that can serve as an intermediate representation for both understanding and genera…
MiLorE-SSL: Scaling Multilingual Capabilities in Self-Supervised Models without Forgetting
Jing Xu, Minglin Wu, Xueyuan Chen +2
Self-supervised learning (SSL) has greatly advanced speech representation learning, but multilingual SSL models remain constrained to languages encountered during pretraining. Retr…
ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extraction
Wenxuan Wu, Shuai Wang, Xixin Wu +2
Audio-visual target speaker extraction (AV-TSE) models primarily rely on visual cues from the target speaker. However, humans also leverage linguistic knowledge, such as syntactic…
Detecting Neurocognitive Disorders through Analyses of Topic Evolution and Cross-modal Consistency in Visual-Stimulated Narratives
Jinchao Li, Yuejiao Wang, Junan Li +14
Early detection of neurocognitive disorders (NCDs) is crucial for timely intervention and disease management. Given that language impairments manifest early in NCD progression, vis…