How Far Are We from Robust Voice Conversion: A Survey

Abstract

Voice conversion technologies have been greatly improved in recent years with the help of deep learning, but their capabilities of producing natural sounding utterances in different conditions remain unclear. In this paper, we gave a thorough study of the robustness of known VC models. We also modified these models, such as the replacement of speaker embeddings, to further improve their performances. We found that the sampling rate and audio duration greatly influence voice conversion. All the VC models suffer from unseen data, but AdaIN-VC is relatively more robust. Also, the speaker embedding jointly trained is more suitable for voice conversion than those trained on speaker identification.

arXiv commentAccepted by SLT 2021

References in corpus (10)

• WaveNet: A Generative Model for Raw Audio
• VoxCeleb2: Deep Speaker Recognition
• VoxCeleb: a large-scale speaker identification dataset
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
• THCHS-30 : A Free Chinese Speech Corpus
• Unsupervised Speech Decomposition via Triple Information Bottleneck
• Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion
• Generative Adversarial Networks for Unpaired Voice Transformation on Impaired Speech
• Defending Your Voice: Adversarial Attack on Voice Conversion