Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classification

Abstract

Several studies have proposed deep-learning-based models to predict the mean opinion score (MOS) of synthesized speech, showing the possibility of replacing human raters. However, inter- and intra-rater variability in MOSs makes it hard to ensure the high performance of the models. In this paper, we propose a multi-task learning (MTL) method to improve the performance of a MOS prediction model using the following two auxiliary tasks: spoofing detection (SD) and spoofing type classification (STC). Besides, we use the focal loss to maximize the synergy between SD and STC for MOS prediction. Experiments using the MOS evaluation results of the Voice Conversion Challenge 2018 show that proposed MTL with two auxiliary tasks improves MOS prediction. Our proposed model achieves up to 11.6% relative improvement in performance over the baseline model.

arXiv comment8 pages, 5 figures, accepted to SLT 2021

References in corpus (6)

• WaveNet: A Generative Model for Raw Audio
• An Overview of Multi-Task Learning in Deep Neural Networks
• MOSNet: Deep Learning based Objective Assessment for Voice Conversion
• AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech
• Deep MOS Predictor for Synthetic Speech Using Cluster-Based Modeling
• Identifying beneficial task relations for multi-task learning in deep neural networks

Cited by in corpus (1)

• MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network