Cross-Lingual Transfer Learning for Question Answering

Abstract

Deep learning based question answering (QA) on English documents has achieved success because there is a large amount of English training examples. However, for most languages, training examples for high-quality QA models are not available. In this paper, we explore the problem of cross-lingual transfer learning for QA, where a source language task with plentiful annotations is utilized to improve the performance of a QA model on a target language task with limited available annotations. We examine two different approaches. A machine translation (MT) based approach translates the source language into the target language, or vice versa. Although the MT-based approach brings improvement, it assumes the availability of a sentence-level translation system. A GAN-based approach incorporates a language discriminator to learn language-universal feature representations, and consequentially transfer knowledge from the source language. The GAN-based approach rivals the performance of the MT-based approach with fewer linguistic resources. Applying both approaches simultaneously yield the best results. We use two English benchmark datasets, SQuAD and NewsQA, as source language data, and show significant improvements over a number of established baselines on a Chinese QA task. We achieve the new state-of-the-art on the Chinese QA dataset.

References in corpus (20)

• Cross-lingual Language Model Pretraining
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Dynamic Coattention Networks For Question Answering
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Enriching Word Vectors with Subword Information
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
• Word Translation Without Parallel Data
• RACE: Large-scale ReAding Comprehension Dataset From Examinations
• DRCD: a Chinese Machine Reading Comprehension Dataset
• NewsQA: A Machine Comprehension Dataset
• Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
• Multilingual Extractive Reading Comprehension by Runtime Machine Translation
• DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• Stochastic Answer Networks for Machine Reading Comprehension
• Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension
• Question Answering through Transfer Learning from Large Fine-grained Supervision Data
• Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog
• Supervised and Unsupervised Transfer Learning for Question Answering
• Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension