10 papers
Attacking UTMOS: Probing the Robustness of a Speech Quality Assessment Model
Wen-Chin Huang, Tomoki Toda
UTMOS has become one of the most commonly used deep neural network-based speech quality assessment (SQA) metrics in speech processing research. In this paper, we attack UTMOS to pr…
RAS: a Reliability Oriented Metric for Automatic Speech Recognition
Wenbin Huang, Yuhang Qiu, Bohan Li +5
Automatic speech recognition systems often produce confident yet incorrect transcriptions under noisy or ambiguous conditions, which can be misleading for both users and downstream…
An Extensive Analysis of the Singing Voice Conversion Challenge 2025 Evaluation Results
Lester Phillip Violeta, Xueyao Zhang, Jiatong Shi +4
We present a thorough analysis of the findings of the latest iteration of the Singing Voice Conversion Challenge, a scientific event aiming to compare and understand different voic…
MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Models
Wen-Chin Huang, Erica Cooper, Tomoki Toda
In this paper, we study the task of subjective speech quality assessment (SSQA), which refers to predicting the perceptual quality of speech. Owing to the development of deep neura…
MOS-Bias: From Hidden Gender Bias to Gender-Aware Speech Quality Assessment
Wenze Ren, Yi-Cheng Lin, Wen-Chin Huang +5
The Mean Opinion Score (MOS) serves as the standard metric for speech quality assessment, yet biases in human annotations remain underexplored. We conduct the first systematic anal…
The AudioMOS Challenge 2025
Wen-Chin Huang, Hui Wang, Cheng Liu +6
This is the summary paper for the AudioMOS Challenge 2025, the very first challenge for automatic subjective quality prediction for synthetic audio. The challenge consists of three…