VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention
arXiv:2102.06431
Abstract
This paper proposes VARA-TTS, a non-autoregressive (non-AR) text-to-speech (TTS) model using a very deep Variational Autoencoder (VDVAE) with Residual Attention mechanism, which refines the textual-to-acoustic alignment layer-wisely. Hierarchical latent variables with different temporal resolutions from the VDVAE are used as queries for residual attention module. By leveraging the coarse global alignment from previous attention layer as an extra input, the following attention layer can produce a refined version of alignment. This amortizes the burden of learning the textual-to-acoustic alignment among multiple attention layers and outperforms the use of only a single attention layer in robustness. An utterance-level speaking speed factor is computed by a jointly-trained speaking speed predictor, which takes the mean-pooled latent variables of the coarsest layer as input, to determine number of acoustic frames at inference. Experimental results show that VARA-TTS achieves slightly inferior speech quality to an AR counterpart Tacotron 2 but an order-of-magnitude speed-up at inference; and outperforms an analogous non-AR model, BVAE-TTS, in terms of speech quality.
References in corpus (15)
- Gaussian Error Linear Units (GELUs)
- Neural Discrete Representation Learning
- Conditional Image Generation with PixelCNN Decoders
- Attention-Based Models for Speech Recognition
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- A Recurrent Latent Variable Model for Sequential Data
- Deep Voice: Real-time Neural Text-to-Speech
- Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention
- Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
- Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis
- Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis
- Efficient Neural Audio Synthesis
- Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
- End-to-End Adversarial Text-to-Speech
- RealFormer: Transformer Likes Residual Attention