Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM

Abstract

We present a state-of-the-art end-to-end Automatic Speech Recognition (ASR) model. We learn to listen and write characters with a joint Connectionist Temporal Classification (CTC) and attention-based encoder-decoder network. The encoder is a deep Convolutional Neural Network (CNN) based on the VGG network. The CTC network sits on top of the encoder and is jointly trained with the attention-based decoder. During the beam search process, we combine the CTC predictions, the attention-based decoder predictions and a separately trained LSTM language model. We achieve a 5-10\% error reduction compared to prior systems on spontaneous Japanese and Chinese speech, and our end-to-end model beats out traditional hybrid ASR systems.

arXiv commentAccepted for INTERSPEECH 2017

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ADADELTA: An Adaptive Learning Rate Method
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks
• Latent Sequence Decompositions

Cited by in corpus (8)

• Monotonic Chunkwise Attention
• Multi-encoder multi-resolution framework for end-to-end speech recognition
• End-to-end Speech Recognition with Word-based RNN Language Models
• Stream attention-based multi-array end-to-end speech recognition
• Language model integration based on memory control for sequence to sequence speech recognition
• Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers
• Acoustic-to-Word Models with Conversational Context Information
• Deep learning networks for selection of persistent scatterer pixels in multi-temporal SAR interferometric processing