Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering

Abstract

Coupled with the availability of large scale datasets, deep learning architectures have enabled rapid progress on the Question Answering task. However, most of those datasets are in English, and the performances of state-of-the-art multilingual models are significantly lower when evaluated on non-English data. Due to high data collection costs, it is not realistic to obtain annotated data for each language one desires to support. We propose a method to improve the Cross-lingual Question Answering performance without requiring additional annotated data, leveraging Question Generation models to produce synthetic samples in a cross-lingual fashion. We show that the proposed method allows to significantly outperform the baselines trained on English data only. We report a new state-of-the-art on four multilingual datasets: MLQA, XQuAD, SQuAD-it and PIAF (fr).

arXiv comment7 pages

References in corpus (9)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Teaching Machines to Read and Comprehend
• Unified Language Model Pre-training for Natural Language Understanding and Generation
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• CamemBERT: a Tasty French Language Model
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
• Project PIAF: Building a Native French Question-Answering Dataset
• Querent Intent in Multi-Sentence Questions

Cited by in corpus (3)

• Data Augmentation Approaches in Natural Language Processing: A Survey
• One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval
• The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfer