Parallel Tacotron: Non-Autoregressive and Controllable TTS

Abstract

Although neural end-to-end text-to-speech models can synthesize highly natural speech, there is still room for improvements to its efficiency and naturalness. This paper proposes a non-autoregressive neural text-to-speech model augmented with a variational autoencoder-based residual encoder. This model, called \emph{Parallel Tacotron}, is highly parallelizable during both training and inference, allowing efficient synthesis on modern parallel hardware. The use of the variational autoencoder relaxes the one-to-many mapping nature of the text-to-speech problem and improves naturalness. To further improve the naturalness, we use lightweight convolutions, which can efficiently capture local contexts, and introduce an iterative spectrogram loss inspired by iterative refinement. Experimental results show that Parallel Tacotron matches a strong autoregressive baseline in subjective evaluations with significantly decreased inference time.

References in corpus (13)

• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• FastSpeech: Fast, Robust and Controllable Text to Speech
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Non-Autoregressive Neural Machine Translation
• Layer Normalization
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• Pay Less Attention with Lightweight and Dynamic Convolutions
• DurIAN: Duration Informed Attention Network For Multimodal Synthesis
• Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling
• End-to-End Adversarial Text-to-Speech
• JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment
• TalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis Model
• Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis

Cited by in corpus (12)

• A Survey on Neural Speech Synthesis
• Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling
• Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech
• Review of end-to-end speech synthesis technology based on deep learning
• Recent Advances and Trends in Multimodal Deep Learning: A Review
• Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling
• FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice Conversion
• Axial Residual Networks for CycleGAN-based Voice Conversion
• VRAIN-UPV MLLP's system for the Blizzard Challenge 2021
• On-device neural speech synthesis
• Sprachsynthese -- State-of-the-Art in englischer und deutscher Sprache
• FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis