Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
arXiv:2004.14069
Abstract
Multilingual pre-trained models could leverage the training data from a rich source language (such as English) to improve performance on low resource languages. However, the transfer quality for multilingual Machine Reading Comprehension (MRC) is significantly worse than sentence classification tasks mainly due to the requirement of MRC to detect the word level answer boundary. In this paper, we propose two auxiliary tasks in the fine-tuning stage to create additional phrase boundary supervision: (1) A mixed MRC task, which translates the question or passage to other languages and builds cross-lingual question-passage pairs; (2) A language-agnostic knowledge masking task by leveraging knowledge phrases mined from web. Besides, extensive experiments on two cross-lingual MRC datasets show the effectiveness of our proposed approach.
Accepted to ACL 2020
References in corpus (20)
- Teaching Machines to Read and Comprehend
- Cross-lingual Language Model Pretraining
- Bidirectional Attention Flow for Machine Comprehension
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Adversarial Examples for Evaluating Reading Comprehension Systems
- XNLI: Evaluating Cross-lingual Sentence Representations
- How multilingual is Multilingual BERT?
- ERNIE: Enhanced Language Representation with Informative Entities
- Multi-Perspective Context Matching for Machine Comprehension
- Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
- ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
- Dynamic Integration of Background Knowledge in Neural NLU Systems
- XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
- Multilingual Extractive Reading Comprehension by Runtime Machine Translation
- Cross-Lingual Machine Reading Comprehension
- Reinforced Mnemonic Reader for Machine Reading Comprehension
- Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing
- Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER
- Knowledge Based Machine Reading Comprehension