SVSNet: An End-to-end Speaker Voice Similarity Assessment Model

Abstract

Neural evaluation metrics derived for numerous speech generation tasks have recently attracted great attention. In this paper, we propose SVSNet, the first end-to-end neural network model to assess the speaker voice similarity between converted speech and natural speech for voice conversion tasks. Unlike most neural evaluation metrics that use hand-crafted features, SVSNet directly takes the raw waveform as input to more completely utilize speech information for prediction. SVSNet consists of encoder, co-attention, distance calculation, and prediction modules and is trained in an end-to-end manner. The experimental results on the Voice Conversion Challenge 2018 and 2020 (VCC2018 and VCC2020) datasets show that SVSNet outperforms well-known baseline systems in the assessment of speaker similarity at the utterance and system levels.

To appear in IEEE Signal Processing Letters (SPL)

References in corpus (6)

• WaveNet: A Generative Model for Raw Audio
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• Deep Learning Based Assessment of Synthetic Speech Naturalness
• Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances
• STOI-Net: A Deep Learning based Non-Intrusive Speech Intelligibility Assessment Model
• A non-causal FFTNet architecture for speech enhancement

Cited by in corpus (1)

• Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features