Choosing Transfer Languages for Cross-Lingual Learning

Abstract

Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages. However, given a particular task language, it is not clear which language to transfer from, and the standard strategy is to select languages based on ad hoc criteria, usually the intuition of the experimenter. Since a large number of features contribute to the success of cross-lingual transfer (including phylogenetic similarity, typological properties, lexical overlap, or size of available data), even the most enlightened experimenter rarely considers all these factors for the particular task at hand. In this paper, we consider this task of automatically selecting optimal transfer languages as a ranking problem, and build models that consider the aforementioned features to perform this prediction. In experiments on representative NLP tasks, we demonstrate that our model predicts good transfer languages much better than ad hoc baselines considering single features in isolation, and glean insights on what features are most informative for each different NLP tasks, which may inform future ad hoc selection even without use of our method. Code, data, and pre-trained models are available at https://github.com/neulab/langrank

Proceedings of ACL 2019

Cited by in corpus (12)

• Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus
• Balancing Training for Multilingual Neural Machine Translation
• Analysing The Impact Of Linguistic Features On Cross-Lingual Transfer
• Generalized Data Augmentation for Low-Resource Translation
• Improving Portuguese Semantic Role Labeling with Transformers and Transfer Learning
• A Survey on Low-Resource Neural Machine Translation
• Improving Candidate Generation for Low-resource Cross-lingual Entity Linking
• MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning
• Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model
• Detecting Foodborne Illness Complaints in Multiple Languages Using English Annotations Only
• Predicting the Performance of Multilingual NLP Models
• Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Machine Translation