Rapid Adaptation of Neural Machine Translation to New Languages

Abstract

This paper examines the problem of adapting neural machine translation systems to new, low-resourced languages (LRLs) as effectively and rapidly as possible. We propose methods based on starting with massively multilingual "seed models", which can be trained ahead-of-time, and then continuing training on data related to the LRL. We contrast a number of strategies, leading to a novel, simple, yet effective method of "similar-language regularization", where we jointly train on both a LRL of interest and a similar high-resourced language to prevent over-fitting to small LRL data. Experiments demonstrate that massively multilingual models, even without any explicit adaptation, are surprisingly effective, achieving BLEU scores of up to 15.5 with no data from the LRL, and that the proposed similar-language regularization method improves over other adaptation methods by 1.7 BLEU points average over 4 LRL settings. Code to reproduce experiments at https://github.com/neubig/rapid-adaptation

arXiv commentEMNLP 2018

References in corpus (7)

• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
• Phrase-Based & Neural Unsupervised Machine Translation
• Unsupervised Machine Translation Using Monolingual Corpora Only
• Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
• Unsupervised Neural Machine Translation
• XNMT: The eXtensible Neural Machine Translation Toolkit
• Universal Neural Machine Translation for Extremely Low Resource Languages

Cited by in corpus (8)

• XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
• A Study of Multilingual Neural Machine Translation
• Robust Optimization for Multilingual Translation with Imbalanced Data
• A Survey on Low-Resource Neural Machine Translation
• Towards a General-Purpose Linguistic Annotation Backend
• Revisiting Modularized Multilingual NMT to Meet Industrial Demands
• Active Learning for Massively Parallel Translation of Constrained Text into Low Resource Languages
• Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation