Fully Convolutional Speech Recognition

Abstract

Current state-of-the-art speech recognition systems build on recurrent neural networks for acoustic and/or language modeling, and rely on feature extraction pipelines to extract mel-filterbanks or cepstral coefficients. In this paper we present an alternative approach based solely on convolutional neural networks, leveraging recent advances in acoustic models from the raw waveform and language modeling. This fully convolutional approach is trained end-to-end to predict characters from the raw waveform, removing the feature extraction step altogether. An external convolutional language model is used to decode words. On Wall Street Journal, our model matches the current state-of-the-art. On Librispeech, we report state-of-the-art performance among end-to-end models, including Deep Speech 2 trained with 12 times more acoustic data and significantly more linguistic data.

References in corpus (9)

• WaveNet: A Generative Model for Raw Audio
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Convolutional Sequence to Sequence Learning
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Listen, Attend and Spell
• Improved training of end-to-end attention models for speech recognition
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• The CAPIO 2017 Conversational Speech Recognition System
• Deep Recurrent Neural Networks for Acoustic Modelling

Cited by in corpus (29)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Self-Training for End-to-End Speech Recognition
• Language Modeling with Deep Transformers
• End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
• Transformers with convolutional context for ASR
• ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?
• QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions
• Who Needs Words? Lexicon-Free Speech Recognition
• Semi-Supervised Speech Recognition via Local Prior Matching
• Iterative Pseudo-Labeling for Speech Recognition
• Pushing the Limits of Non-Autoregressive Speech Recognition
• A Fully Differentiable Beam Search Decoder
• Espresso: A Fast End-to-end Neural Speech Recognition Toolkit
• CAT: CRF-based ASR Toolkit
• FluentNet: End-to-End Detection of Speech Disfluency with Deep Learning
• Do End-to-End Speech Recognition Models Care About Context?
• Convolutional Speech Recognition with Pitch and Voice Quality Features
• Efficient Neural Architecture Search for End-to-end Speech Recognition via Straight-Through Gradients
• Deformable TDNN with adaptive receptive fields for speech recognition
• KoSpeech: Open-Source Toolkit for End-to-End Korean Speech Recognition
• Back from the future: bidirectional CTC decoding using future information in speech recognition
• Attention-based ASR with Lightweight and Dynamic Convolutions
• Multi-QuartzNet: Multi-Resolution Convolution for Speech Recognition with Multi-Layer Feature Fusion
• CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latency
• PyChain: A Fully Parallelized PyTorch Implementation of LF-MMI for End-to-End ASR
• Sequential Routing Framework: Fully Capsule Network-based Speech Recognition
• A Parallelizable Lattice Rescoring Strategy with Neural Language Models