Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embedding
arXiv:2005.08484
Abstract
On account of growing demands for personalization, the need for a so-called few-shot TTS system that clones speakers with only a few data is emerging. To address this issue, we propose Attentron, a few-shot TTS model that clones voices of speakers unseen during training. It introduces two special encoders, each serving different purposes. A fine-grained encoder extracts variable-length style information via an attention mechanism, and a coarse-grained encoder greatly stabilizes the speech synthesis, circumventing unintelligible gibberish even for synthesizing speech of unseen speakers. In addition, the model can scale out to an arbitrary number of reference audios to improve the quality of the synthesized speech. According to our experiments, including a human evaluation, the proposed model significantly outperforms state-of-the-art models when generating speech for unseen speakers in terms of speaker similarity and quality.
Accepted to INTERSPEECH 2020
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Sample Efficient Adaptive Text-to-Speech
- ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
- Effective Use of Variational Embedding Capacity in Expressive End-to-End Speech Synthesis
- Hierarchical Generative Modeling for Controllable Speech Synthesis
- Modeling Multi-speaker Latent Space to Improve Neural TTS: Quick Enrolling New Speaker and Enhancing Premium Voice
- Multi-Speaker End-to-End Speech Synthesis
- Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings
- Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis