ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit

Abstract

This paper introduces a new end-to-end text-to-speech (E2E-TTS) toolkit named ESPnet-TTS, which is an extension of the open-source speech processing toolkit ESPnet. The toolkit supports state-of-the-art E2E-TTS models, including Tacotron~2, Transformer TTS, and FastSpeech, and also provides recipes inspired by the Kaldi automatic speech recognition (ASR) toolkit. The recipes are based on the design unified with the ESPnet ASR recipe, providing high reproducibility. The toolkit also provides pre-trained models and samples of all of the recipes so that users can use it as a baseline. Furthermore, the unified design enables the integration of ASR functions with TTS, e.g., ASR-based objective evaluation and semi-supervised learning with both ASR and TTS models. This paper describes the design of the toolkit and experimental evaluation in comparison with other toolkits. The experimental results show that our models can achieve state-of-the-art performance comparable to the other latest toolkits, resulting in a mean opinion score (MOS) of 4.25 on the LJSpeech dataset. The toolkit is publicly available at https://github.com/espnet/espnet.

arXiv commentAccepted to ICASSP2020. Demo HP: https://espnet.github.io/icassp2020-tts/

References in corpus (12)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Attention-Based Models for Speech Recognition
• A Comparative Study on Transformer vs RNN in Speech Applications
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention
• Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
• Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
• JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis
• Forward Attention in Sequence-to-sequence Acoustic Modelling for Speech Synthesis
• Almost Unsupervised Text to Speech and Automatic Speech Recognition
• JVS corpus: free Japanese multi-speaker voice corpus
• Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq

Cited by in corpus (3)

• Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining
• Pretraining Techniques for Sequence-to-Sequence Voice Conversion
• Articulatory-WaveNet: Autoregressive Model For Acoustic-to-Articulatory Inversion