XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering

Abstract

While natural language processing systems often focus on a single language, multilingual transfer learning has the potential to improve performance, especially for low-resource languages. We introduce XLDA, cross-lingual data augmentation, a method that replaces a segment of the input text with its translation in another language. XLDA enhances performance of all 14 tested languages of the cross-lingual natural language inference (XNLI) benchmark. With improvements of up to $4.8\%$, training with XLDA achieves state-of-the-art performance for Greek, Turkish, and Urdu. XLDA is in contrast to, and performs markedly better than, a more naive approach that aggregates examples in various languages in a way that each example is solely in one language. On the SQuAD question answering task, we see that XLDA provides a $1.0\%$ performance increase on the English evaluation set. Comprehensive experiments suggest that most languages are effective as cross-lingual augmentors, that XLDA is robust to a wide range of translation quality, and that XLDA is even more effective for randomly initialized models than for pretrained models.

References in corpus (13)

• Distributed Representations of Words and Phrases and their Compositionality
• Cross-lingual Language Model Pretraining
• Exploiting Similarities among Languages for Machine Translation
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
• Word Translation Without Parallel Data
• XNLI: Evaluating Cross-lingual Sentence Representations
• Multilingual Models for Compositional Distributed Semantics
• Multilingual Extractive Reading Comprehension by Runtime Machine Translation
• Universal Neural Machine Translation for Extremely Low Resource Languages
• Rapid Adaptation of Neural Machine Translation to New Languages

Cited by in corpus (10)

• Translation Artifacts in Cross-lingual Transfer Learning
• Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretraining
• Neural Semi-supervised Learning for Text Classification Under Large-Scale Pretraining
• LAReQA: Language-agnostic answer retrieval from a multilingual pool
• Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
• Consistency Regularization for Cross-Lingual Fine-Tuning
• BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation
• Knowing More About Questions Can Help: Improving Calibration in Question Answering
• Linguistic Knowledge in Data Augmentation for Natural Language Processing: An Example on Chinese Question Matching