PidginUNMT: Unsupervised Neural Machine Translation from West African Pidgin to English
arXiv:1912.03444
Abstract
Over 800 languages are spoken across West Africa. Despite the obvious diversity among people who speak these languages, one language significantly unifies them all - West African Pidgin English. There are at least 80 million speakers of West African Pidgin English. However, there is no known natural language processing (NLP) work on this language. In this work, we perform the first NLP work on the most popular variant of the language, providing three major contributions. First, the provision of a Pidgin corpus of over 56000 sentences, which is the largest we know of. Secondly, the training of the first ever cross-lingual embedding between Pidgin and English. This aligned embedding will be helpful in the performance of various downstream tasks between English and Pidgin. Thirdly, the training of an Unsupervised Neural Machine Translation model between Pidgin and English which achieves BLEU scores of 7.93 from Pidgin to English, and 5.18 from English to Pidgin. In all, this work greatly reduces the barrier of entry for future NLP works on West African Pidgin English.
arXiv commentPresented at NeurIPS 2019 Workshop on Machine Learning for the Developing World
References in corpus (8)
- Adam: A Method for Stochastic Optimization
- Exploiting Similarities among Languages for Machine Translation
- Massively Multilingual Word Embeddings
- Word Translation Without Parallel Data
- Phrase-Based & Neural Unsupervised Machine Translation
- Unsupervised Machine Translation Using Monolingual Corpora Only
- Unsupervised Neural Machine Translation
- Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion