Adversarial Learning of Intermediate Acoustic Feature for End-to-End Lightweight Text-to-Speech
arXiv:2204.02172 · doi:10.21437/Interspeech.2023-1571
Abstract
To simplify the generation process, several text-to-speech (TTS) systems implicitly learn intermediate latent representations instead of relying on predefined features (e.g., mel-spectrogram). However, their generation quality is unsatisfactory as these representations lack speech variances. In this paper, we improve TTS performance by adding \emph{prosody embeddings} to the latent representations. During training, we extract reference prosody embeddings from mel-spectrograms, and during inference, we estimate these embeddings from text using generative adversarial networks (GANs). Using GANs, we reliably estimate the prosody embeddings in a fast way, which have complex distributions due to the dynamic nature of speech. We also show that the prosody embeddings work as efficient features for learning a robust alignment between text and acoustic features. Our proposed model surpasses several publicly available models with less parameters and computational complexity in comparative experiments.
INTERSPEECH 2023
References in corpus (6)
- WaveNet: A Generative Model for Raw Audio
- HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
- FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
- SpeechBrain: A General-Purpose Speech Toolkit
- Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search
- LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation