15 papers
Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs
Trung Nguyen Quang, Cheng Yi Lewis Won, Minh Duc Pham +3
Audio large language models (Audio LLMs) exhibit systematic failures in transcribing code-switching speech despite strong multilingual capabilities. Focusing on English-Mandarin, w…
Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance
Weihua Zheng, Chang Liu, Zhengyuan Liu +5
Multilingual Large Language Models (LLMs) struggle with cross-lingual tasks due to data imbalances between high-resource and low-resource languages, as well as monolingual bias in…
Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-off
Longyin Zhang, Shuo Sun, Yingxu He +15
Recent advancements in Multimodal Large Language Models (MLLMs) pursue omni-perception capabilities, yet integrating robust sensory grounding with complex reasoning remains a chall…
MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
Hardik B. Sailor, Aw Ai Ti, Chen Fang Yih Nancy +26
We present MERaLiON-SER, a robust speech emotion recognition model designed for English and Southeast Asian languages. The model is trained using a hybrid objective combining weigh…
IFEval-Audio: Benchmarking Instruction-Following Capability in Audio-based Large Language Models
Yiming Gao, Bin Wang, Chengwei Wei +2
Large language models (LLMs) have demonstrated strong instruction-following capabilities in text-based tasks. However, this ability often deteriorates in multimodal models after al…
Beyond Classification: Towards Speech Emotion Reasoning with Multitask AudioLLMs
Wenyu Zhang, Yingxu He, Geyu Lin +9
Audio Large Language Models (AudioLLMs) have achieved strong results in semantic tasks like speech recognition and translation, but remain limited in modeling paralinguistic cues s…