Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
arXiv:1809.06963
Abstract
We propose a multi-task learning framework to learn a joint Machine Reading Comprehension (MRC) model that can be applied to a wide range of MRC tasks in different domains. Inspired by recent ideas of data selection in machine translation, we develop a novel sample re-weighting scheme to assign sample-specific weights to the loss. Empirical study shows that our approach can be applied to many existing MRC models. Combined with contextual representations from pre-trained language models (such as ELMo), we achieve new state-of-the-art results on a set of MRC benchmark datasets. We release our code at https://github.com/xycforgithub/MultiTask-MRC.
arXiv commentNorth American Chapter of the Association for Computational Linguistics (NAACL) 2019
References in corpus (19)
- Adam: A Method for Stochastic Optimization
- Natural Language Processing (almost) from Scratch
- An Overview of Multi-Task Learning in Deep Neural Networks
- Bidirectional Attention Flow for Machine Comprehension
- A Structured Self-attentive Sentence Embedding
- Exploring the Limits of Language Modeling
- Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
- Regularizing and Optimizing LSTM Language Models
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- The Natural Language Decathlon: Multitask Learning as Question Answering
- ReasoNet: Learning to Stop Reading in Machine Comprehension
- Multi-Task Deep Neural Networks for Natural Language Understanding
- Reading Wikipedia to Answer Open-Domain Questions
- Stochastic Answer Networks for Natural Language Inference
- Words or Characters? Fine-grained Gating for Reading Comprehension
- Who did What: A Large-Scale Person-Centered Cloze Dataset
- Dynamic Fusion Networks for Machine Reading Comprehension
- Read + Verify: Machine Reading Comprehension with Unanswerable Questions
- Stochastic Answer Networks for SQuAD 2.0
Cited by in corpus (5)
- Multi-Task Deep Neural Networks for Natural Language Understanding
- Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
- Stochastic Answer Networks for Natural Language Inference
- Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
- Stochastic Answer Networks for SQuAD 2.0