ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context

Abstract

Convolutional neural networks (CNN) have shown promising results for end-to-end speech recognition, albeit still behind other state-of-the-art methods in performance. In this paper, we study how to bridge this gap and go beyond with a novel CNN-RNN-transducer architecture, which we call ContextNet. ContextNet features a fully convolutional encoder that incorporates global context information into convolution layers by adding squeeze-and-excitation modules. In addition, we propose a simple scaling method that scales the widths of ContextNet that achieves good trade-off between computation and accuracy. We demonstrate that on the widely used LibriSpeech benchmark, ContextNet achieves a word error rate (WER) of 2.1%/4.6% without external language model (LM), 1.9%/4.1% with LM and 2.9%/7.0% with only 10M parameters on the clean/noisy LibriSpeech test sets. This compares to the previous best published system of 2.0%/4.6% with LM and 3.9%/11.3% with 20M parameters. The superiority of the proposed ContextNet model is also verified on a much larger internal dataset.

Submitted to Interspeech 2020

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Sequence Transduction with Recurrent Neural Networks
• A Comparative Study on Transformer vs RNN in Speech Applications
• Searching for Activation Functions
• Transformer-based Acoustic Modeling for Hybrid Speech Recognition
• Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
• Fully Convolutional Speech Recognition
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions
• RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions
• Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models

Cited by in corpus (24)

• SpeechBrain: A General-Purpose Speech Toolkit
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition
• Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition
• A New Training Pipeline for an Improved Neural Transducer
• Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
• Fine-grained Style Modeling, Transfer and Prediction in Text-to-Speech Synthesis via Phone-Level Content-Style Disentanglement
• Pushing the Limits of Non-Autoregressive Speech Recognition
• Self-training and Pre-training are Complementary for Speech Recognition
• PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
• Efficient conformer-based speech recognition with linear attention
• Improved Conformer-based End-to-End Speech Recognition Using Neural Architecture Search
• FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization
• "Train one, Classify one, Teach one" -- Cross-surgery transfer learning for surgical step recognition
• One In A Hundred: Select The Best Predicted Sequence from Numerous Candidates for Streaming Speech Recognition
• Improving RNN transducer with normalized jointer network
• ASAPP-ASR: Multistream CNN and Self-Attentive SRU for SOTA Speech Recognition
• Efficient End-to-End Speech Recognition Using Performers in Conformers
• Stability of Neural Networks on Riemannian Manifolds
• KoSpeech: Open-Source Toolkit for End-to-End Korean Speech Recognition
• Streaming Transformer ASR with Blockwise Synchronous Beam Search
• Multitask Learning and Joint Optimization for Transformer-RNN-Transducer Speech Recognition
• GIPFA: Generating IPA Pronunciation from Audio
• CarneliNet: Neural Mixture Model for Automatic Speech Recognition