ESPnet: End-to-End Speech Processing Toolkit

Abstract

This paper introduces a new open source platform for end-to-end speech processing named ESPnet. ESPnet mainly focuses on end-to-end automatic speech recognition (ASR), and adopts widely-used dynamic neural network toolkits, Chainer and PyTorch, as a main deep learning engine. ESPnet also follows the Kaldi ASR toolkit style for data processing, feature extraction/format, and recipes to provide a complete setup for speech recognition and other speech processing experiments. This paper explains a major architecture of this software platform, several important functionalities, which differentiate ESPnet from other open source ASR toolkits, and experimental results with major ASR benchmarks.

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding
• Get To The Point: Summarization with Pointer-Generator Networks

Cited by in corpus (47)

• wav2letter++: The Fastest Open-source Speech Recognition System
• Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends
• DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation
• End-to-end training of time domain audio separation and recognition
• UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
• Data Augmentation for End-to-end Code-switching Speech Recognition
• ESPnet2-TTS: Extending the Edge of TTS Research
• Fine-grained Style Modeling, Transfer and Prediction in Text-to-Speech Synthesis via Phone-Level Content-Style Disentanglement
• The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Results
• WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit
• On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition
• An improved hybrid CTC-Attention model for speech recognition
• Multitask-Based Joint Learning Approach To Robust ASR For Radio Communication Speech
• Universal ASR: Unifying Streaming and Non-Streaming ASR Using a Single Encoder-Decoder Model
• WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition
• IMS-Speech: A Speech to Text Tool
• End-to-end Speech Recognition with Word-based RNN Language Models
• Towards Semi-Supervised Semantics Understanding from Speech
• AISPEECH-SJTU accent identification system for the Accented English Speech Recognition Challenge
• Semantic-WER: A Unified Metric for the Evaluation of ASR Transcript for End Usability
• CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition
• Synchronous Transformers for End-to-End Speech Recognition
• Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling
• Cross-Attention End-to-End ASR for Two-Party Conversations
• fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit
• Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units
• Improving Speech Enhancement Performance by Leveraging Contextual Broad Phonetic Class Information
• PDAugment: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics Transcription
• End-to-End Monaural Multi-speaker ASR System without Pretraining
• Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping
• Efficient End-to-End Speech Recognition Using Performers in Conformers
• Promising Accurate Prefix Boosting for sequence-to-sequence ASR
• Deep Discriminative Feature Learning for Accent Recognition
• Language model integration based on memory control for sequence to sequence speech recognition
• Cycle-consistency training for end-to-end speech recognition
• Dialog-context aware end-to-end speech recognition
• LSSED: a large-scale dataset and benchmark for speech emotion recognition
• Homophone-based Label Smoothing in End-to-End Automatic Speech Recognition
• MusicTM-Dataset for Joint Representation Learning among Sheet Music, Lyrics, and Musical Audio
• Distortion-controlled Training for End-to-end Reverberant Speech Separation with Auxiliary Autoencoding Loss
• End-to-end contextual speech recognition using class language models and a token passing decoder
• Loss Prediction: End-to-End Active Learning Approach For Speech Recognition
• Improving End-to-end Speech Recognition with Pronunciation-assisted Sub-word Modeling
• ON-TRAC Consortium End-to-End Speech Translation Systems for the IWSLT 2019 Shared Task
• Gender domain adaptation for automatic speech recognition task
• Performance Monitoring for End-to-End Speech Recognition
• Acoustic-to-Word Models with Conversational Context Information