14 papers
MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages
Qiongqiong Wang, Ai Ti Aw, Nancy F. Chen +18
We present MERaLiON-GR, a speech gender recognition system that performs binary classification (female / male) on English and Southeast Asian (SEA) languages. The model finetunes M…
Improving Code-Switching ASR with Code-Mixing Guided Synthetic Speech
Yue Heng Yeo, Haoyang Li, Yizhou Peng +6
Code-switch (CS) Automatic Speech Recognition (ASR) remains challenging due to limited availability of high quality CS text-speech pairs for training. Although synthetic data augme…
Goodness-of-pronunciation without phoneme time alignment
Jeremy H. M. Wong, Nancy F. Chen
In speech evaluation, an Automatic Speech Recognition (ASR) model often computes time boundaries and phoneme posteriors for input features. However, limited data for ASR training h…
Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-off
Longyin Zhang, Shuo Sun, Yingxu He +15
Recent advancements in Multimodal Large Language Models (MLLMs) pursue omni-perception capabilities, yet integrating robust sensory grounding with complex reasoning remains a chall…
MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
Hardik B. Sailor, Aw Ai Ti, Chen Fang Yih Nancy +26
We present MERaLiON-SER, a robust speech emotion recognition model designed for English and Southeast Asian languages. The model is trained using a hybrid objective combining weigh…
Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models
Qiongqiong Wang, Hardik B. Sailor, Jeremy H. M. Wong +6
Current large speech language models (Speech-LLMs) often exhibit limitations in empathetic reasoning, primarily due to the absence of training datasets that integrate both contextu…
ASTAR-NTU solution to AudioMOS Challenge 2025 Track1
Fabian Ritter-Gutierrez, Yi-Cheng Lin, Jui-Chiang Wei +3
Evaluation of text-to-music systems is constrained by the cost and availability of collecting experts for assessment. AudioMOS 2025 Challenge track 1 is created to automatically pr…
A correlation-permutation approach for speech-music encoders model merging
Fabian Ritter-Gutierrez, Yi-Cheng Lin, Jeremy H. M Wong +3
Creating a unified speech and music model requires expensive pre-training. Model merging can instead create an unified audio model with minimal computational expense. However, dire…
Beyond Classification: Towards Speech Emotion Reasoning with Multitask AudioLLMs
Wenyu Zhang, Yingxu He, Geyu Lin +9
Audio Large Language Models (AudioLLMs) have achieved strong results in semantic tasks like speech recognition and translation, but remain limited in modeling paralinguistic cues s…
Distilling a speech and music encoder with task arithmetic
Fabian Ritter-Gutierrez, Yi-Cheng Lin, Jui-Chiang Wei +4
Despite the progress in self-supervised learning (SSL) for speech and music, existing models treat these domains separately, limiting their capacity for unified audio understanding…
MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyond
Muhammad Huzaifah, Geyu Lin, Tianchi Liu +8
This technical report describes the MERaLiON-SpeechEncoder, a foundation model designed to support a wide range of downstream speech applications. Developed as part of Singapore's…
Semi-supervised Learning For Robust Speech Evaluation
Huayun Zhang, Jeremy H. M. Wong, Geyu Lin +1
Speech evaluation measures a learners oral proficiency using automatic models. Corpora for training such models often pose sparsity challenges given that there often is limited sco…
Dataset-Distillation Generative Model for Speech Emotion Recognition
Fabian Ritter-Gutierrez, Kuan-Po Huang, Jeremy H. M Wong +4
Deep learning models for speech rely on large datasets, presenting computational challenges. Yet, performance hinges on training data size. Dataset Distillation (DD) aims to learn…
Noise robust distillation of self-supervised speech models via correlation metrics
Fabian Ritter-Gutierrez, Kuan-Po Huang, Dianwen Ng +4
Compared to large speech foundation models, small distilled models exhibit degraded noise robustness. The student's robustness can be improved by introducing noise at the inputs du…