On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition

Abstract

Code-switching (CS) refers to a linguistic phenomenon where a speaker uses different languages in an utterance or between alternating utterances. In this work, we study end-to-end (E2E) approaches to the Mandarin-English code-switching speech recognition (CSSR) task. We first examine the effectiveness of using data augmentation and byte-pair encoding (BPE) subword units. More importantly, we propose a multitask learning recipe, where a language identification task is explicitly learned in addition to the E2E speech recognition task. Furthermore, we introduce an efficient word vocabulary expansion method for language modeling to alleviate data sparsity issues under the code-switching scenario. Experimental results on the SEAME data, a Mandarin-English CS corpus, demonstrate the effectiveness of the proposed methods.

arXiv commentAccepted for Interspeech 2019

References in corpus (5)

• Attention-Based Models for Speech Recognition
• Improved training of end-to-end attention models for speech recognition
• First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs
• ESPnet: End-to-End Speech Processing Toolkit
• Unsupervised and Efficient Vocabulary Expansion for Recurrent Neural Network Language Models in ASR

Cited by in corpus (5)

• A Survey of Code-switched Speech and Language Processing
• Semi-supervised Development of ASR Systems for Multilingual Code-switched Speech in Under-resourced Languages
• Semi-supervised acoustic modelling for five-lingual code-switched ASR using automatically-segmented soap opera speech
• Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition
• Semi-supervised acoustic and language model training for English-isiZulu code-switched speech recognition