Improving Cross-Lingual Reading Comprehension with Self-Training

Abstract

Substantial improvements have been made in machine reading comprehension, where the machine answers questions based on a given context. Current state-of-the-art models even surpass human performance on several benchmarks. However, their abilities in the cross-lingual scenario are still to be explored. Previous works have revealed the abilities of pre-trained multilingual models for zero-shot cross-lingual reading comprehension. In this paper, we further utilized unlabeled data to improve the performance. The model is first supervised-trained on source language corpus, and then self-trained with unlabeled target language data. The experiment results showed improvements for all languages, and we also analyzed how self-training benefits cross-lingual reading comprehension in qualitative aspects.

arXiv comment8 pages, 4 figures

References in corpus (10)

• Decoupled Weight Decay Regularization
• Language Models are Few-Shot Learners
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• DRCD: a Chinese Machine Reading Comprehension Dataset
• Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation
• Self-training Improves Pre-training for Natural Language Understanding
• Cross-Lingual Machine Reading Comprehension
• KorQuAD1.0: Korean QA Dataset for Machine Reading Comprehension
• Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model
• FQuAD: French Question Answering Dataset