Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis

Abstract

In this paper we propose Flowtron: an autoregressive flow-based generative network for text-to-speech synthesis with control over speech variation and style transfer. Flowtron borrows insights from IAF and revamps Tacotron in order to provide high-quality and expressive mel-spectrogram synthesis. Flowtron is optimized by maximizing the likelihood of the training data, which makes training simple and stable. Flowtron learns an invertible mapping of data to a latent space that can be manipulated to control many aspects of speech synthesis (pitch, tone, speech rate, cadence, accent). Our mean opinion scores (MOS) show that Flowtron matches state-of-the-art TTS models in terms of speech quality. In addition, we provide results on control of speech variation, interpolation between samples and style transfer between speakers seen and unseen during training. Code and pre-trained models will be made publicly available at https://github.com/NVIDIA/flowtron

10 pages, 7 pictures

References in corpus (4)

• NICE: Non-linear Independent Components Estimation
• Deep Voice: Real-time Neural Text-to-Speech
• High Fidelity Speech Synthesis with Adversarial Networks
• Transflow Learning: Repurposing Flow Models Without Retraining

Cited by in corpus (10)

• A Survey on Neural Speech Synthesis
• Review of end-to-end speech synthesis technology based on deep learning
• VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention
• Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
• EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture
• Speech Synthesis and Control Using Differentiable DSP
• A Survey on Audio Synthesis and Audio-Visual Multimodal Processing
• GANtron: Emotional Speech Synthesis with Generative Adversarial Networks
• Text-to-speech for the hearing impaired
• Sprachsynthese -- State-of-the-Art in englischer und deutscher Sprache