The 2020 ESPnet update: new features, broadened applications, performance improvements, and future plans

Abstract

This paper describes the recent development of ESPnet (https://github.com/espnet/espnet), an end-to-end speech processing toolkit. This project was initiated in December 2017 to mainly deal with end-to-end speech recognition experiments based on sequence-to-sequence modeling. The project has grown rapidly and now covers a wide range of speech processing applications. Now ESPnet also includes text to speech (TTS), voice conversation (VC), speech translation (ST), and speech enhancement (SE) with support for beamforming, speech separation, denoising, and dereverberation. All applications are trained in an end-to-end manner, thanks to the generic sequence to sequence modeling properties, and they can be further integrated and jointly optimized. Also, ESPnet provides reproducible all-in-one recipes for these applications with state-of-the-art performance in various benchmarks by incorporating transformer, advanced data augmentation, and conformer. This project aims to provide up-to-date speech processing experience to the community so that researchers in academia and various industry scales can develop their technologies collaboratively.

References in corpus (13)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Sequence to Sequence Learning with Neural Networks
• Attention-Based Models for Speech Recognition
• Sequence Transduction with Recurrent Neural Networks
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
• CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition
• Recent Developments on ESPnet Toolkit Boosted by Conformer
• Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition
• Espresso: A Fast End-to-end Neural Speech Recognition Toolkit
• End-to-End Speech Recognition: A review for the French Language
• Directional ASR: A New Paradigm for E2E Multi-Speaker Speech Recognition with Source Localization

Cited by in corpus (2)

• Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement
• A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion