MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension
arXiv:2009.14348
Abstract
Span extraction is an essential problem in machine reading comprehension. Most of the existing algorithms predict the start and end positions of an answer span in the given corresponding context by generating two probability vectors. In this paper, we propose a novel approach that extends the probability vector to a probability matrix. Such a matrix can cover more start-end position pairs. Precisely, to each possible start index, the method always generates an end probability vector. Besides, we propose a sampling-based training strategy to address the computational cost and memory issue in the matrix training phase. We evaluate our method on SQuAD 1.1 and three other question answering benchmarks. Leveraging the most competitive models BERT and BiDAF as the backbone, our proposed approach can get consistent improvements in all datasets, demonstrating the effectiveness of the proposed method.
to appear at AACL-IJCNLP 2020
References in corpus (8)
- ADADELTA: An Adaptive Learning Rate Method
- XLNet: Generalized Autoregressive Pretraining for Language Understanding
- Bidirectional Attention Flow for Machine Comprehension
- Learning Recurrent Span Representations for Extractive Question Answering
- SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
- Neural Machine Reading Comprehension: Methods and Trends
- Unifying Question Answering, Text Classification, and Regression via Span Extraction
- Contextual Aware Joint Probability Model Towards Question Answering System