Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation

Abstract

We present a simple method to improve neural translation of a low-resource language pair using parallel data from a related, also low-resource, language pair. The method is based on the transfer method of Zoph et al., but whereas their method ignores any source vocabulary overlap, ours exploits it. First, we split words using Byte Pair Encoding (BPE) to increase vocabulary overlap. Then, we train a model on the first language pair and transfer its parameters, including its source word embeddings, to another model and continue training on the second language pair. Our experiments show that transfer learning helps word-based translation only slightly, but when used on top of a much stronger BPE baseline, it yields larger improvements of up to 4.3 BLEU.

References in corpus (1)

• ADADELTA: An Adaptive Learning Rate Method

Cited by in corpus (12)

• Multilingual Neural Machine Translation With Soft Decoupled Encoding
• Low-resource Languages: A Review of Past Work and Future Challenges
• Low-Resource Neural Machine Translation for Southern African Languages
• Adapting Multilingual Neural Machine Translation to Unseen Languages
• Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation
• Translating the Unseen? Yoruba-English MT in Low-Resource, Morphologically-Unmarked Settings
• A Survey on Low-Resource Neural Machine Translation
• Contextual Text Embeddings for Twi
• Unsupervised Spoken Term Discovery on Untranscribed Speech
• Cross-lingual Word Embeddings beyond Zero-shot Machine Translation
• Predicting the Performance of Multilingual NLP Models
• Self-Learning for Zero Shot Neural Machine Translation