Towards achieving robust universal neural vocoding

Abstract

This paper explores the potential universality of neural vocoders. We train a WaveRNN-based vocoder on 74 speakers coming from 17 languages. This vocoder is shown to be capable of generating speech of consistently good quality (98% relative mean MUSHRA when compared to natural speech) regardless of whether the input spectrogram comes from a speaker or style seen during training or from an out-of-domain scenario when the recording conditions are studio-quality. When the recordings show significant changes in quality, or when moving towards non-speech vocalizations or singing, the vocoder still significantly outperforms speaker-dependent vocoders, but operates at a lower average relative MUSHRA of 75%. These results are shown to be consistent across languages, regardless of them being seen during training (e.g. English or Japanese) or unseen (e.g. Wolof, Swahili, Ahmaric).

arXiv comment4 pages, 1 extra for references. Accepted on Interspeech 2019

References in corpus (5)

• WaveNet: A Generative Model for Raw Audio
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
• Efficient Neural Audio Synthesis
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech

Cited by in corpus (5)

• Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains
• Towards Robust Neural Vocoding for Speech Generation: A Survey
• TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguese
• Attention Forcing for Sequence-to-sequence Model Training
• Reverberation Modeling for Source-Filter-based Neural Vocoder