Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains

Abstract

We propose Universal MelGAN, a vocoder that synthesizes high-fidelity speech in multiple domains. To preserve sound quality when the MelGAN-based structure is trained with a dataset of hundreds of speakers, we added multi-resolution spectrogram discriminators to sharpen the spectral resolution of the generated waveforms. This enables the model to generate realistic waveforms of multi-speakers, by alleviating the over-smoothing problem in the high frequency band of the large footprint model. Our structure generates signals close to ground-truth data without reducing the inference speed, by discriminating the waveform and spectrogram during training. The model achieved the best mean opinion score (MOS) in most scenarios using ground-truth mel-spectrogram as an input. Especially, it showed superior performance in unseen domains with regard of speaker, emotion, and language. Moreover, in a multi-speaker text-to-speech scenario using mel-spectrogram generated by a transformer model, it synthesized high-fidelity speech of 4.22 MOS. These results, achieved without external domain information, highlight the potential of the proposed model as a universal vocoder.

References in corpus (7)

• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• DurIAN: Duration Informed Attention Network For Multimodal Synthesis
• Efficient Neural Audio Synthesis
• Towards Robust Neural Vocoding for Speech Generation: A Survey
• Direct speech-to-speech translation with a sequence-to-sequence model
• JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment
• Towards achieving robust universal neural vocoding

Cited by in corpus (8)

• A Survey on Neural Speech Synthesis
• Fre-GAN: Adversarial Frequency-consistent Audio Synthesis
• Neural Synthesis of Footsteps Sound Effects with Generative Adversarial Networks
• GAN Vocoder: Multi-Resolution Discriminator Is All You Need
• Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder For High Fidelity Flow-based Speech Synthesis
• AdaVocoder: Adaptive Vocoder for Custom Voice
• Basis-MelGAN: Efficient Neural Vocoder Based on Audio Decomposition
• Towards Universal Neural Vocoding with a Multi-band Excited WaveNet