Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search

Abstract

Recently, text-to-speech (TTS) models such as FastSpeech and ParaNet have been proposed to generate mel-spectrograms from text in parallel. Despite the advantage, the parallel TTS models cannot be trained without guidance from autoregressive TTS models as their external aligners. In this work, we propose Glow-TTS, a flow-based generative model for parallel TTS that does not require any external aligner. By combining the properties of flows and dynamic programming, the proposed model searches for the most probable monotonic alignment between text and the latent representation of speech on its own. We demonstrate that enforcing hard monotonic alignments enables robust TTS, which generalizes to long utterances, and employing generative flows enables fast, diverse, and controllable speech synthesis. Glow-TTS obtains an order-of-magnitude speed-up over the autoregressive model, Tacotron 2, at synthesis with comparable speech quality. We further show that our model can be easily extended to a multi-speaker setting.

arXiv commentAccepted by NeurIPS2020

References in corpus (11)

• WaveNet: A Generative Model for Raw Audio
• Neural Discrete Representation Learning
• NICE: Non-linear Independent Components Estimation
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Non-Autoregressive Neural Machine Translation
• Sequence-Level Knowledge Distillation
• Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis
• Efficient Neural Audio Synthesis
• Emerging Convolutions for Generative Normalizing Flows
• Non-Autoregressive Neural Text-to-Speech
• Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework

Cited by in corpus (41)

• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
• AdaSpeech: Adaptive Text to Speech for Custom Voice
• VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature
• The Ethical Implications of Generative Audio Models: A Systematic Literature Review
• End-to-End Adversarial Text-to-Speech
• Transfer Learning Framework for Low-Resource Text-to-Speech using a Large-Scale Unlabeled Speech Corpus
• UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding
• WavThruVec: Latent speech representation as intermediate features for neural speech synthesis
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
• SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech
• Heterogeneous Target Speech Separation
• TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguese
• Coarse-to-Fine Proposal Refinement Framework for Audio Temporal Forgery Detection and Localization
• Prosody-controllable spontaneous TTS with neural HMMs
• VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention
• Giving Robots a Voice: Human-in-the-Loop Voice Creation and open-ended Labeling
• Neural HMMs are all you need (for high-quality attention-free TTS)
• LightGrad: Lightweight Diffusion Probabilistic Model for Text-to-Speech
• Creating New Voices using Normalizing Flows
• Music Source Separation with Generative Flow
• Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
• SNAC: Speaker-normalized affine coupling layer in flow-based architecture for zero-shot multi-speaker text-to-speech
• OverFlow: Putting flows on top of neural transducers for better TTS
• Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAE
• Mitigating Unauthorized Speech Synthesis for Voice Protection
• ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
• PortaSpeech: Portable and High-Quality Generative Text-to-Speech
• Speech Synthesis and Control Using Differentiable DSP
• A Survey on Audio Synthesis and Audio-Visual Multimodal Processing
• EdiTTS: Score-based Editing for Controllable Text-to-Speech
• Preference-based training framework for automatic speech quality assessment using deep neural network
• Neural Dubber: Dubbing for Videos According to Scripts
• Personalized One-Shot Lipreading for an ALS Patient
• Embedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimization
• Rhythm-controllable Attention with High Robustness for Long Sentence Speech Synthesis
• EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusion
• Adversarial Learning of Intermediate Acoustic Feature for End-to-End Lightweight Text-to-Speech
• Varianceflow: High-Quality and Controllable Text-to-Speech using Variance Information via Normalizing Flow
• Pruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speech
• Sprachsynthese -- State-of-the-Art in englischer und deutscher Sprache
• AI based Presentation Creator With Customized Audio Content Delivery