ESPnet2-TTS: Extending the Edge of TTS Research
arXiv:2110.07840
Abstract
This paper describes ESPnet2-TTS, an end-to-end text-to-speech (E2E-TTS) toolkit. ESPnet2-TTS extends our earlier version, ESPnet-TTS, by adding many new features, including: on-the-fly flexible pre-processing, joint training with neural vocoders, and state-of-the-art TTS models with extensions like full-band E2E text-to-waveform modeling, which simplify the training pipeline and further enhance TTS performance. The unified design of our recipes enables users to quickly reproduce state-of-the-art E2E-TTS results. We also provide many pre-trained models in a unified Python interface for inference, offering a quick means for users to generate baseline samples and build demos. Experimental evaluations with English and Japanese corpora demonstrate that our provided models synthesize utterances comparable to ground-truth ones, achieving state-of-the-art TTS performance. The toolkit is available online at https://github.com/espnet/espnet.
arXiv commentSubmitted to ICASSP2022. Demo HP: https://espnet.github.io/icassp2022-tts/
References in corpus (13)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
- MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
- Conformer: Convolution-augmented Transformer for Speech Recognition
- Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention
- NeMo: a toolkit for building AI applications using Neural Modules
- Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech
- High Fidelity Speech Synthesis with Adversarial Networks
- JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis
- ESPnet: End-to-End Speech Processing Toolkit
- JVS corpus: free Japanese multi-speaker voice corpus
- Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining
- MultiSpeech: Multi-Speaker Text to Speech with Transformer