Exploring Neural Transducers for End-to-End Speech Recognition
arXiv:1707.07413
Abstract
In this work, we perform an empirical comparison among the CTC, RNN-Transducer, and attention-based Seq2Seq models for end-to-end speech recognition. We show that, without any language model, Seq2Seq and RNN-Transducer models both outperform the best reported CTC models with a language model, on the popular Hub5'00 benchmark. On our internal diverse dataset, these trends continue - RNNTransducer models rescored with a language model after beam search outperform our best CTC models. These results simplify the speech recognition pipeline so that decoding can now be expressed purely as neural network operations. We also study how the choice of encoder architecture affects the performance of the three models - when all encoder layers are forward only, and when encoders downsample the input representation aggressively.
References in corpus (7)
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Sequence Transduction with Recurrent Neural Networks
- Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
- First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs
- Towards better decoding and language model integration in sequence to sequence models
- Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition
- Reducing Bias in Production Speech Models
Cited by in corpus (5)
- SirenAttack: Generating Adversarial Audio for End-to-End Acoustic Systems
- Improving the Performance of Online Neural Transducer Models
- Listen, Attend, Spell and Adapt: Speaker Adapted Sequence-to-Sequence ASR
- End-To-End Speech Recognition Using A High Rank LSTM-CTC Based Model
- Speaker Adaptation for End-to-End CTC Models