Code-switching Language Modeling With Bilingual Word Embeddings: A Case Study for Egyptian Arabic-English
arXiv:1909.10892
Abstract
Code-switching (CS) is a widespread phenomenon among bilingual and multilingual societies. The lack of CS resources hinders the performance of many NLP tasks. In this work, we explore the potential use of bilingual word embeddings for code-switching (CS) language modeling (LM) in the low resource Egyptian Arabic-English language. We evaluate different state-of-the-art bilingual word embeddings approaches that require cross-lingual resources at different levels and propose an innovative but simple approach that jointly learns bilingual word representations without the use of any parallel data, relying only on monolingual and a small amount of CS data. While all representations improve CS LM, ours performs the best and improves perplexity 33.5% relative over the baseline.
arXiv comment11 pages, 1 figure (having 2 sub-figures), submitted to the 21st International Conference on Speech and Computer (SPECOM'19),
References in corpus (4)
- Distributed Representations of Words and Phrases and their Compositionality
- BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
- TheanoLM - An Extensible Toolkit for Neural Network Language Modeling
- A Novel Bilingual Word Embedding Method for Lexical Translation Using Bilingual Sense Clique