Towards Robust Neural Vocoding for Speech Generation: A Survey

Abstract

Recently, neural vocoders have been widely used in speech synthesis tasks, including text-to-speech and voice conversion. However, when encountering data distribution mismatch between training and inference, neural vocoders trained on real data often degrade in voice quality for unseen scenarios. In this paper, we train four common neural vocoders, including WaveNet, WaveRNN, FFTNet, Parallel WaveGAN alternately on five different datasets. To study the robustness of neural vocoders, we evaluate the models using acoustic features from seen/unseen speakers, seen/unseen languages, a text-to-speech model, and a voice conversion model. We found out that the speaker variety is much more important for achieving a universal vocoder than the language. Through our experiments, we show that WaveNet and WaveRNN are more suitable for text-to-speech models, while Parallel WaveGAN is more suitable for voice conversion applications. Great amount of subjective MOS results in naturalness for all vocoders are presented for future studies.

arXiv commentSubmitted to INTERSPEECH 2020

References in corpus (10)

• WaveNet: A Generative Model for Raw Audio
• Conditional Image Generation with PixelCNN Decoders
• FastSpeech: Fast, Robust and Controllable Text to Speech
• AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• Tacotron: Towards End-to-End Speech Synthesis
• Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
• Efficient Neural Audio Synthesis
• Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion
• Towards achieving robust universal neural vocoding

Cited by in corpus (6)

• A Survey on Neural Speech Synthesis
• Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains
• Parallel Synthesis for Autoregressive Speech Generation
• Can Speaker Augmentation Improve Multi-Speaker End-to-End TTS?
• AdaVocoder: Adaptive Vocoder for Custom Voice
• Sprachsynthese -- State-of-the-Art in englischer und deutscher Sprache