Data Augmentation for End-to-end Code-switching Speech Recognition

Abstract

Training a code-switching end-to-end automatic speech recognition (ASR) model normally requires a large amount of data, while code-switching data is often limited. In this paper, three novel approaches are proposed for code-switching data augmentation. Specifically, they are audio splicing with the existing code-switching data, and TTS with new code-switching texts generated by word translation or word insertion. Our experiments on 200 hours Mandarin-English code-switching dataset show that all the three proposed approaches yield significant improvements on code-switching ASR individually. Moreover, all the proposed approaches can be combined with recent popular SpecAugment, and an addition gain can be obtained. WER is significantly reduced by relative 24.0% compared to the system without any data augmentation, and still relative 13.0% gain compared to the system with only SpecAugment

Accepted to SLT 2021

References in corpus (10)

• Auto-Encoding Variational Bayes
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• A Comparative Study on Transformer vs RNN in Speech Applications
• FastSpeech: Fast, Robust and Controllable Text to Speech
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• ESPnet: End-to-End Speech Processing Toolkit
• Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data
• Towards End-to-end Automatic Code-Switching Speech Recognition
• word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairs
• Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition

Cited by in corpus (5)

• On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASR
• Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation Models
• QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpus
• Semantic Data Augmentation for End-to-End Mandarin Speech Recognition
• Global Structure-Aware Drum Transcription Based on Self-Attention Mechanisms