Letter-Based Speech Recognition with Gated ConvNets

Abstract

In the recent literature, "end-to-end" speech systems often refer to letter-based acoustic models trained in a sequence-to-sequence manner, either via a recurrent model or via a structured output learning approach (such as CTC). In contrast to traditional phone (or senone)-based approaches, these "end-to-end'' approaches alleviate the need of word pronunciation modeling, and do not require a "forced alignment" step at training time. Phone-based approaches remain however state of the art on classical benchmarks. In this paper, we propose a letter-based speech recognition system, leveraging a ConvNet acoustic model. Key ingredients of the ConvNet are Gated Linear Units and high dropout. The ConvNet is trained to map audio sequences to their corresponding letter transcriptions, either via a classical CTC approach, or via a recent variant called ASG. Coupled with a simple decoder at inference time, our system matches the best existing letter-based systems on WSJ (in word error rate), and shows near state of the art performance on LibriSpeech.

arXiv comment13 pages.arXiv admin note: text overlap with arXiv:1609.03193

References in corpus (11)

• On the difficulty of training Recurrent Neural Networks
• Convolutional Sequence to Sequence Learning
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• The Microsoft 2016 Conversational Speech Recognition System
• Improved training of end-to-end attention models for speech recognition
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs
• Towards better decoding and language model integration in sequence to sequence models
• Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling
• Deep Recurrent Neural Networks for Acoustic Modelling
• End-to-End Neural Segmental Models for Speech Recognition

Cited by in corpus (16)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Libri-Light: A Benchmark for ASR with Limited or No Supervision
• MLS: A Large-Scale Multilingual Dataset for Speech Research
• Improved training of end-to-end attention models for speech recognition
• Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition
• wav2letter++: The Fastest Open-source Speech Recognition System
• Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions
• From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition
• CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition
• MarbleNet: Deep 1D Time-Channel Separable Convolutional Neural Network for Voice Activity Detection
• Optimal Completion Distillation for Sequence Learning
• Unpaired Image-to-Speech Synthesis with Multimodal Information Bottleneck
• CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcription
• Multi-Frame Cross-Entropy Training for Convolutional Neural Networks in Speech Recognition