Multichannel End-to-end Speech Recognition

Abstract

The field of speech recognition is in the midst of a paradigm shift: end-to-end neural networks are challenging the dominance of hidden Markov models as a core technology. Using an attention mechanism in a recurrent encoder-decoder architecture solves the dynamic time alignment problem, allowing joint end-to-end training of the acoustic and language modeling components. In this paper we extend the end-to-end framework to encompass microphone array signal processing for noise suppression and speech enhancement within the acoustic encoding network. This allows the beamforming components to be optimized jointly within the recognition architecture to improve the end-to-end speech recognition objective. Experiments on the noisy speech benchmarks (CHiME-4 and AMI) show that our multichannel end-to-end system outperformed the attention-based baseline with input from a conventional adaptive beamformer.

References in corpus (5)

• On the difficulty of training Recurrent Neural Networks
• Attention-Based Models for Speech Recognition
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding
• Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition

Cited by in corpus (16)

• Frequency Domain Multi-channel Acoustic Modeling for Distant Speech Recognition
• End-to-End Far-Field Speech Recognition with Unified Dereverberation and Beamforming
• Multi-Geometry Spatial Acoustic Modeling for Distant Speech Recognition
• An Investigation of End-to-End Multichannel Speech Recognition for Reverberant and Mismatch Conditions
• End-to-End Multi-speaker Speech Recognition with Transformer
• Directional ASR: A New Paradigm for E2E Multi-Speaker Speech Recognition with Source Localization
• MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition
• End-to-End Dereverberation, Beamforming, and Speech Recognition with Improved Numerical Stability and Advanced Frontend
• Fully Learnable Front-End for Multi-Channel Acoustic Modeling using Semi-Supervised Learning
• Consistency-aware multi-channel speech enhancement using deep neural networks
• Exploring Optimal DNN Architecture for End-to-End Beamformers Based on Time-frequency References
• Exploiting Single-Channel Speech For Multi-channel End-to-end Speech Recognition
• Improving RNN Transducer With Target Speaker Extraction and Neural Uncertainty Estimation
• Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-field Speech Recognition
• Microphone Array Generalization for Multichannel Narrowband Deep Speech Enhancement
• Far-Field Automatic Speech Recognition