Learn to Code-Switch: Data Augmentation using Copy Mechanism on Language Modeling

Abstract

Building large-scale datasets for training code-switching language models is challenging and very expensive. To alleviate this problem using parallel corpus has been a major workaround. However, existing solutions use linguistic constraints which may not capture the real data distribution. In this work, we propose a novel method for learning how to generate code-switching sentences from parallel corpora. Our model uses a Seq2Seq model in combination with pointer networks to align and choose words from the monolingual sentences and form a grammatical code-switching sentence. In our experiment, we show that by training a language model using the augmented sentences we improve the perplexity score by 10% compared to the LSTM baseline.

arXiv commentSubmitted to ICASSP 2019

References in corpus (1)

• Effective Approaches to Attention-based Neural Machine Translation

Cited by in corpus (5)

• Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data
• Code Switching Language Model Using Monolingual Training Data
• From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text
• HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text
• Training a code-switching language model with monolingual data