Adversarial Learning of Intermediate Acoustic Feature for End-to-End Lightweight Text-to-Speech
arXiv:2204.02172 · doi:10.21437/Interspeech.2023-1571
Abstract
To simplify the generation process, several text-to-speech (TTS) systems implicitly learn intermediate latent representations instead of relying on predefined features (e.g., mel-spectrogram). However, their generation quality is unsatisfactory as these representations lack speech variances. In this paper, we improve TTS performance by adding \emph{prosody embeddings} to the latent representations. During training, we extract reference prosody embeddings from mel-spectrograms, and during inference, we estimate these embeddings from text using generative adversarial networks (GANs). Using GANs, we reliably estimate the prosody embeddings in a fast way, which have complex distributions due to the dynamic nature of speech. We also show that the prosody embeddings work as efficient features for learning a robust alignment between text and acoustic features. Our proposed model surpasses several publicly available models with less parameters and computational complexity in comparative experiments.
arXiv commentINTERSPEECH 2023
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- WaveNet: A Generative Model for Raw Audio
- HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
- FastSpeech: Fast, Robust and Controllable Text to Speech
- FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
- SpeechBrain: A General-Purpose Speech Toolkit
- cGANs with Projection Discriminator
- Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
- LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation