Exploring Neural Transducers for End-to-End Speech Recognition

Abstract

In this work, we perform an empirical comparison among the CTC, RNN-Transducer, and attention-based Seq2Seq models for end-to-end speech recognition. We show that, without any language model, Seq2Seq and RNN-Transducer models both outperform the best reported CTC models with a language model, on the popular Hub5'00 benchmark. On our internal diverse dataset, these trends continue - RNNTransducer models rescored with a language model after beam search outperform our best CTC models. These results simplify the speech recognition pipeline so that decoding can now be expressed purely as neural network operations. We also study how the choice of encoder architecture affects the performance of the three models - when all encoder layers are forward only, and when encoders downsample the input representation aggressively.

References in corpus (12)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Sequence Transduction with Recurrent Neural Networks
• Achieving Human Parity in Conversational Speech Recognition
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding
• First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs
• Towards better decoding and language model integration in sequence to sequence models
• Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition
• Segmental Recurrent Neural Networks for End-to-end Speech Recognition
• Reducing Bias in Production Speech Models
• Morphological Inflection Generation with Hard Monotonic Attention

Cited by in corpus (37)

• Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq
• On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition
• NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations
• Imputer: Sequence Modelling via Imputation and Dynamic Programming
• SirenAttack: Generating Adversarial Audio for End-to-End Acoustic Systems
• Pushing the Limits of Non-Autoregressive Speech Recognition
• A Comparison of Label-Synchronous and Frame-Synchronous End-to-End Models for Speech Recognition
• The Architectural Implications of Facebook's DNN-based Personalized Recommendation
• OmniNet: A unified architecture for multi-modal multi-task learning
• Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset
• A study of latent monotonic attention variants
• End-to-End Speech Recognition: A review for the French Language
• Transfer Learning Approaches for Streaming End-to-End Speech Recognition System
• Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict
• An Online Attention-based Model for Speech Recognition
• Improving the Performance of Online Neural Transducer Models
• Advancing Connectionist Temporal Classification With Attention Modeling
• Contextual Density Ratio for Language Model Biasing of Sequence to Sequence ASR Systems
• Advancing Acoustic-to-Word CTC Model with Attention and Mixed-Units
• Recent Progresses in Deep Learning based Acoustic Models (Updated)
• Semi-Supervised Learning with Data Augmentation for End-to-End ASR
• Listen, Attend, Spell and Adapt: Speaker Adapted Sequence-to-Sequence ASR
• Automatic Grammar Augmentation for Robust Voice Command Recognition
• Tree-constrained Pointer Generator for End-to-end Contextual Speech Recognition
• Neural Image Compression and Explanation
• Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition
• Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models
• Cross-attention conformer for context modeling in speech enhancement for ASR
• Deep Shallow Fusion for RNN-T Personalization
• SNRi Target Training for Joint Speech Enhancement and Recognition
• CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latency
• End-To-End Speech Recognition Using A High Rank LSTM-CTC Based Model
• AV Taris: Online Audio-Visual Speech Recognition
• Training for Speech Recognition on Coprocessors
• Speaker Adaptation for End-to-End CTC Models