Semi-supervised ASR by End-to-end Self-training

Abstract

While deep learning based end-to-end automatic speech recognition (ASR) systems have greatly simplified modeling pipelines, they suffer from the data sparsity issue. In this work, we propose a self-training method with an end-to-end system for semi-supervised ASR. Starting from a Connectionist Temporal Classification (CTC) system trained on the supervised data, we iteratively generate pseudo-labels on a mini-batch of unsupervised utterances with the current model, and use the pseudo-labels to augment the supervised data for immediate model update. Our method retains the simplicity of end-to-end ASR systems, and can be seen as performing alternating optimization over a well-defined learning objective. We also perform empirical investigations of our method, regarding the effect of data augmentation, decoding beamsize for pseudo-label generation, and freshness of pseudo-labels. On a commonly used semi-supervised ASR setting with the WSJ corpus, our method gives 14.4% relative WER improvement over a carefully-trained base system with data augmentation, reducing the performance gap between the base system and the oracle system by 50%.

Accepted by Interspeech 2020

References in corpus (10)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Attention-Based Models for Speech Recognition
• Unsupervised Data Augmentation for Consistency Training
• Sequence Transduction with Recurrent Neural Networks
• Improved Noisy Student Training for Automatic Speech Recognition
• Self-Training for End-to-End Speech Recognition
• Almost Unsupervised Text to Speech and Automatic Speech Recognition
• Iterative Pseudo-Labeling for Speech Recognition
• Machine Speech Chain with One-shot Speaker Adaptation
• Improved Regularization Techniques for End-to-End Speech Recognition

Cited by in corpus (5)

• Iterative Pseudo-Labeling for Speech Recognition
• Towards Semi-Supervised Semantics Understanding from Speech
• Boosting Active Learning for Speech Recognition with Noisy Pseudo-labeled Samples
• Semi-Supervised Learning with Data Augmentation for End-to-End ASR
• Exploiting Large-scale Teacher-Student Training for On-device Acoustic Models