8 papers
The VoiceMOS Challenge 2026: Evaluating Speech Enhancement, Emotional TTS and Accented TTS Systems
Wen-Chin Huang, Wei Wang, Marvin Sach +4
We present the results of the VoiceMOS Challenge 2026, the fifth edition of a scientific challenge on automatic prediction of subjective speech assessments. After expanding the sco…
MOS-Bias: From Hidden Gender Bias to Gender-Aware Speech Quality Assessment
Wenze Ren, Yi-Cheng Lin, Wen-Chin Huang +5
The Mean Opinion Score (MOS) serves as the standard metric for speech quality assessment, yet biases in human annotations remain underexplored. We conduct the first systematic anal…
CodecMOS-Accent: A MOS Benchmark of Resynthesized and TTS Speech from Neural Codecs Across English Accents
Wen-Chin Huang, Nicholas Sanders, Erica Cooper
We present the CodecMOS-Accent dataset, a mean opinion score (MOS) benchmark designed to evaluate neural audio codec (NAC) models and the large language model (LLM)-based text-to-s…
Layer-wise Analysis for Quality of Multilingual Synthesized Speech
Erica Cooper, Takuma Okamoto, Yamato Ohtani +2
While supervised quality predictors for synthesized speech have demonstrated strong correlations with human ratings, their requirement for in-domain labeled training data hinders t…
The AudioMOS Challenge 2025
Wen-Chin Huang, Hui Wang, Cheng Liu +6
This is the summary paper for the AudioMOS Challenge 2025, the very first challenge for automatic subjective quality prediction for synthetic audio. The challenge consists of three…
HighRateMOS: Sampling-Rate Aware Modeling for Speech Quality Assessment
Wenze Ren, Yi-Cheng Lin, Wen-Chin Huang +9
Modern speech quality prediction models are trained on audio data resampled to a specific sampling rate. When faced with higher-rate audio at test time, these models can produce bi…
SHEET: A Multi-purpose Open-source Speech Human Evaluation Estimation Toolkit
Wen-Chin Huang, Erica Cooper, Tomoki Toda
We introduce SHEET, a multi-purpose open-source toolkit designed to accelerate subjective speech quality assessment (SSQA) research. SHEET stands for the Speech Human Evaluation Es…
MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Models
Wen-Chin Huang, Erica Cooper, Tomoki Toda
In this paper, we study the task of subjective speech quality assessment (SSQA), which refers to predicting the perceptual quality of speech. Owing to the development of deep neura…