CarneliNet: Neural Mixture Model for Automatic Speech Recognition

Abstract

End-to-end automatic speech recognition systems have achieved great accuracy by using deeper and deeper models. However, the increased depth comes with a larger receptive field that can negatively impact model performance in streaming scenarios. We propose an alternative approach that we call Neural Mixture Model. The basic idea is to introduce a parallel mixture of shallow networks instead of a very deep network. To validate this idea we design CarneliNet -- a CTC-based neural network composed of three mega-blocks. Each mega-block consists of multiple parallel shallow sub-networks based on 1D depthwise-separable convolutions. We evaluate the model on LibriSpeech, MLS and AISHELL-2 datasets and achieved close to state-of-the-art results for CTC-based models. Finally, we demonstrate that one can dynamically reconfigure the number of parallel sub-networks to accommodate the computational requirements without retraining.

arXiv commentSubmitted to ASRU 2021

References in corpus (20)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• SGDR: Stochastic Gradient Descent with Warm Restarts
• Sequence Transduction with Recurrent Neural Networks
• Conformer: Convolution-augmented Transformer for Speech Recognition
• MLS: A Large-Scale Multilingual Dataset for Speech Research
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scale
• NeMo: a toolkit for building AI applications using Neural Modules
• Transformers with convolutional context for ASR
• Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks
• ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context
• Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• Citrinet: Closing the Gap between Non-Autoregressive and Autoregressive End-to-End Models for Automatic Speech Recognition
• Towards Online End-to-end Transformer Automatic Speech Recognition
• U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition
• A Comparison of Label-Synchronous and Frame-Synchronous End-to-End Models for Speech Recognition
• Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT
• Improving Noise Robustness of an End-to-End Neural Model for Automatic Speech Recognition