Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces
arXiv:2005.09150
Abstract
In this work, we first show that on the widely used LibriSpeech benchmark, our transformer-based context-dependent connectionist temporal classification (CTC) system produces state-of-the-art results. We then show that using wordpieces as modeling units combined with CTC training, we can greatly simplify the engineering pipeline compared to conventional frame-based cross-entropy training by excluding all the GMM bootstrapping, decision tree building and force alignment steps, while still achieving very competitive word-error-rate. Additionally, using wordpieces as modeling units can significantly improve runtime efficiency since we can use larger stride without losing accuracy. We further confirm these findings on two internal VideoASR datasets: German, which is similar to English as a fusional language, and Turkish, which is an agglutinative language.
arXiv commentIn proceedings Interspeech 2020
References in corpus (10)
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
- Sequence Transduction with Recurrent Neural Networks
- A Comparative Study on Transformer vs RNN in Speech Applications
- Transformer-based Acoustic Modeling for Hybrid Speech Recognition
- Language Modeling with Deep Transformers
- End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
- Feedforward Sequential Memory Neural Networks without Recurrent Feedback
- Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units