Wav2Letter: an End-to-End ConvNet-based Speech Recognition System

Abstract

This paper presents a simple end-to-end model for speech recognition, combining a convolutional network based acoustic model and a graph decoding. It is trained to output letters, with transcribed speech, without the need for force alignment of phonemes. We introduce an automatic segmentation criterion for training from sequence annotation without alignment that is on par with CTC while being simpler. We show competitive results in word error rate on the Librispeech corpus with MFCC features, and promising results from raw waveform.

arXiv comment8 pages, 4 figures (7 plots/schemas), 2 tables (4 tabulars)

References in corpus (3)

• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding
• Very Deep Multilingual Convolutional Neural Networks for LVCSR

Cited by in corpus (80)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Deep Audio-Visual Speech Recognition
• MLS: A Large-Scale Multilingual Dataset for Speech Research
• vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
• Improved training of end-to-end attention models for speech recognition
• Transformer-based Acoustic Modeling for Hybrid Speech Recognition
• Self-Training for End-to-End Speech Recognition
• wav2letter++: The Fastest Open-source Speech Recognition System
• End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
• Multimodal Machine Learning: A Survey and Taxonomy
• Transformers with convolutional context for ASR
• Sharp Minima Can Generalize For Deep Nets
• Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
• Effectiveness of self-supervised pre-training for speech recognition
• Fully Convolutional Speech Recognition
• Towards Goal-Oriented Semantic Signal Processing: Applications and Future Challenges
• CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
• Unsupervised Automatic Speech Recognition: A Review
• Exploring Neural Transducers for End-to-End Speech Recognition
• CrypTen: Secure Multi-Party Computation Meets Machine Learning
• Gram-CTC: Automatic Unit Selection and Target Decomposition for Sequence Labelling
• Dense Prediction on Sequences with Time-Dilated Convolutions for Speech Recognition
• Jasper: An End-to-End Convolutional Neural Acoustic Model
• Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
• Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq
• Training Neural Speech Recognition Systems with Synthetic Speech Augmentation
• Letter-Based Speech Recognition with Gated ConvNets
• End-to-End Neural Segmental Models for Speech Recognition
• Who Needs Words? Lexicon-Free Speech Recognition
• Imputer: Sequence Modelling via Imputation and Dynamic Programming
• Trajectory Space Factorization for Deep Video-Based 3D Human Pose Estimation
• Why does CTC result in peaky behavior?
• A Fully Differentiable Beam Search Decoder
• Espresso: A Fast End-to-end Neural Speech Recognition Toolkit
• Learning Robust and Multilingual Speech Representations
• Improved Regularization Techniques for End-to-End Speech Recognition
• Reducing Bias in Production Speech Models
• An investigation of phone-based subword units for end-to-end speech recognition
• Differentiable Weighted Finite-State Transducers
• Attention-based Wav2Text with Feature Transfer Learning
• CAT: CRF-based ASR Toolkit
• Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps
• INT8 Winograd Acceleration for Conv1D Equipped ASR Models Deployed on Mobile Devices
• Improving End-to-End Speech Recognition with Policy Learning
• Intelligent Video Editing: Incorporating Modern Talking Face Generation Algorithms in a Video Editor
• Simplified End-to-End MMI Training and Voting for ASR
• The 2020 ESPnet update: new features, broadened applications, performance improvements, and future plans
• Beyond $L_p$ clipping: Equalization-based Psychoacoustic Attacks against ASRs
• Exploring spectro-temporal features in end-to-end convolutional neural networks
• End-to-End ASR for Code-switched Hindi-English Speech
• A systematic comparison of grapheme-based vs. phoneme-based label units for encoder-decoder-attention models
• Effects of Number of Filters of Convolutional Layers on Speech Recognition Model Accuracy
• Romanian Speech Recognition Experiments from the ROBIN Project
• Optimal Completion Distillation for Sequence Learning
• Convolutional Speech Recognition with Pitch and Voice Quality Features
• Data Techniques For Online End-to-end Speech Recognition
• On the Inductive Bias of Word-Character-Level Multi-Task Learning for Speech Recognition
• Utilizing Domain Knowledge in End-to-End Audio Processing
• Sequence Prediction with Neural Segmental Models
• Learning Noise-Invariant Representations for Robust Speech Recognition
• Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units
• EASTER: Efficient and Scalable Text Recognizer
• Weakly Supervised Construction of ASR Systems with Massive Video Data
• Automatic Grammar Augmentation for Robust Voice Command Recognition
• End-to-end Whispered Speech Recognition with Frequency-weighted Approaches and Pseudo Whisper Pre-training
• MASRI-HEADSET: A Maltese Corpus for Speech Recognition
• EasyASR: A Distributed Machine Learning Platform for End-to-end Automatic Speech Recognition
• An Improved Model for Voicing Silent Speech
• Linguistic Search Optimization for Deep Learning Based LVCSR
• Brain2Char: A Deep Architecture for Decoding Text from Brain Recordings
• End-to-End Language Identification using Multi-Head Self-Attention and 1D Convolutional Neural Networks
• Understanding Recurrent Neural State Using Memory Signatures
• Guided contrastive self-supervised pre-training for automatic speech recognition
• Multi-QuartzNet: Multi-Resolution Convolution for Speech Recognition with Multi-Layer Feature Fusion
• CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latency
• CarneliNet: Neural Mixture Model for Automatic Speech Recognition
• Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models
• Gradient-Adjusted Neuron Activation Profiles for Comprehensive Introspection of Convolutional Speech Recognition Models
• Exploiting Nontrivial Connectivity for Automatic Speech Recognition
• Multi-Modal Transformers Utterance-Level Code-Switching Detection