Residual Convolutional CTC Networks for Automatic Speech Recognition

Abstract

Deep learning approaches have been widely used in Automatic Speech Recognition (ASR) and they have achieved a significant accuracy improvement. Especially, Convolutional Neural Networks (CNNs) have been revisited in ASR recently. However, most CNNs used in existing work have less than 10 layers which may not be deep enough to capture all human speech signal information. In this paper, we propose a novel deep and wide CNN architecture denoted as RCNN-CTC, which has residual connections and Connectionist Temporal Classification (CTC) loss function. RCNN-CTC is an end-to-end system which can exploit temporal and spectral structures of speech signals simultaneously. Furthermore, we introduce a CTC-based system combination, which is different from the conventional frame-wise senone-based one. The basic subsystems adopted in the combination are different types and thus mutually complementary to each other. Experimental results show that our proposed single system RCNN-CTC can achieve the lowest word error rate (WER) on WSJ and Tencent Chat data sets, compared to several widely used neural network systems in ASR. In addition, the proposed system combination can offer a further error reduction on these two data sets, resulting in relative WER reductions of $14.91\%$ and $6.52\%$ on WSJ dev93 and Tencent Chat data sets respectively.

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Wide Residual Networks
• EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding
• Deep Recurrent Convolutional Neural Network: Improving Performance For Speech Recognition
• Very Deep Convolutional Neural Networks for Robust Speech Recognition
• Very Deep Convolutional Networks for End-to-End Speech Recognition

Cited by in corpus (22)

• Deep Audio-Visual Speech Recognition
• Adversarial Weight Perturbation Helps Robust Generalization
• End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
• Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
• Improving Adversarial Robustness via Channel-wise Activation Suppressing
• Adversarial Neuron Pruning Purifies Backdoored Deep Models
• Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
• Multi-encoder multi-resolution framework for end-to-end speech recognition
• Offline Handwritten Chinese Text Recognition with Convolutional Neural Networks
• Improving Query Efficiency of Black-box Adversarial Attack
• Simplified End-to-End MMI Training and Voting for ASR
• Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
• Adversarial Training Makes Weight Loss Landscape Sharper in Logistic Regression
• Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillation
• Multi-view Frequency LSTM: An Efficient Frontend for Automatic Speech Recognition
• Multi-Stream End-to-End Speech Recognition
• Clustering Effect of (Linearized) Adversarial Robust Models
• Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training
• Convolutional Attention-based Seq2Seq Neural Network for End-to-End ASR
• Two-stage Training for Chinese Dialect Recognition
• Temporal Calibrated Regularization for Robust Noisy Label Learning