11 papers
Few-Shot and Pseudo-Label Guided Speech Quality Evaluation with Large Language Models
Ryandhimas E. Zezario, Dyah A. M. G. Wisnu, Szu-Wei Fu +3
In this paper, we introduce GatherMOS, a novel framework that leverages large language models (LLM) as meta-evaluators to aggregate diverse signals into quality predictions. Gather…
SAVe: Self-Supervised Audio-visual Deepfake Detection Exploiting Visual Artifacts and Audio-visual Misalignment
Sahibzada Adil Shahzad, Ammarah Hashmi, Junichi Yamagishi +5
Multimodal deepfakes can exhibit subtle visual artifacts and cross-modal inconsistencies, which remain challenging to detect, especially when detectors are trained primarily on cur…
MOS-Bias: From Hidden Gender Bias to Gender-Aware Speech Quality Assessment
Wenze Ren, Yi-Cheng Lin, Wen-Chin Huang +5
The Mean Opinion Score (MOS) serves as the standard metric for speech quality assessment, yet biases in human annotations remain underexplored. We conduct the first systematic anal…
LLM-Guided Reinforcement Learning for Audio-Visual Speech Enhancement
Chih-Ning Chen, Jen-Cheng Hou, Hsin-Min Wang +3
In existing Audio-Visual Speech Enhancement (AVSE) methods, objectives such as Scale-Invariant Signal-to-Noise Ratio (SI-SNR) and Mean Squared Error (MSE) are widely used; however,…
STSM-FiLM: A FiLM-Conditioned Neural Architecture for Time-Scale Modification of Speech
Dyah A. M. G. Wisnu, Ryandhimas E. Zezario, Stefano Rini +4
Time-Scale Modification (TSM) of speech aims to alter the playback rate of audio without changing its pitch. While classical methods like Waveform Similarity-based Overlap-Add (WSO…
Speech Intelligibility Assessment with Uncertainty-Aware Whisper Embeddings and sLSTM
Ryandhimas E. Zezario, Dyah A. M. G. Wisnu, Hsin-Min Wang +1
Non-intrusive speech intelligibility prediction remains challenging due to variability in speakers, noise conditions, and subjective perception. We propose an uncertainty-aware app…
Improving Perceptual Audio Aesthetic Assessment via Triplet Loss and Self-Supervised Embeddings
Dyah A. M. G. Wisnu, Ryandhimas E. Zezario, Stefano Rini +2
We present a system for automatic multi-axis perceptual quality prediction of generative audio, developed for Track 2 of the AudioMOS Challenge 2025. The task is to predict four Au…
A Study on Zero-Shot Non-Intrusive Speech Intelligibility for Hearing Aids Using Large Language Models
Ryandhimas E. Zezario, Dyah A. M. G. Wisnu, Hsin-Min Wang +1
This work focuses on zero-shot non-intrusive speech assessment for hearing aids (HA) using large language models (LLMs). Specifically, we introduce GPT-Whisper-HA, an extension of…
HighRateMOS: Sampling-Rate Aware Modeling for Speech Quality Assessment
Wenze Ren, Yi-Cheng Lin, Wen-Chin Huang +9
Modern speech quality prediction models are trained on audio data resampled to a specific sampling rate. When faced with higher-rate audio at test time, these models can produce bi…
A Comparative Study on Proactive and Passive Detection of Deepfake Speech
Chia-Hua Wu, Wanying Ge, Xin Wang +3
Solutions for defending against deepfake speech fall into two categories: proactive watermarking models and passive conventional deepfake detectors. While both address common threa…
The VoiceMOS Challenge 2024: Beyond Speech Quality Prediction
Wen-Chin Huang, Szu-Wei Fu, Erica Cooper +5
We present the third edition of the VoiceMOS Challenge, a scientific initiative designed to advance research into automatic prediction of human speech ratings. There were three tra…