Preference-based training framework for automatic speech quality assessment using deep neural network

Abstract

One objective of Speech Quality Assessment (SQA) is to estimate the ranks of synthetic speech systems. However, recent SQA models are typically trained using low-precision direct scores such as mean opinion scores (MOS) as the training objective, which is not straightforward to estimate ranking. Although it is effective for predicting quality scores of individual sentences, this approach does not account for speech and system preferences when ranking multiple systems. We propose a training framework of SQA models that can be trained with only preference scores derived from pairs of MOS to improve ranking prediction. Our experiment reveals conditions where our framework works the best in terms of pair generation, aggregation functions to derive system score from utterance preferences, and threshold functions to determine preference from a pair of MOS. Our results demonstrate that our proposed method significantly outperforms the baseline model in Spearman's Rank Correlation Coefficient.

arXiv commentAccepted by Interspeech 2023, oral

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
• Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder
• NORESQA: A Framework for Speech Quality Assessment using Non-Matching References
• When is it Better to Compare than to Score?
• LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech