Read + Verify: Machine Reading Comprehension with Unanswerable Questions
arXiv:1808.05759
Abstract
Machine reading comprehension with unanswerable questions aims to abstain from answering when no answer can be inferred. In addition to extract answers, previous works usually predict an additional "no-answer" probability to detect unanswerable cases. However, they fail to validate the answerability of the question by verifying the legitimacy of the predicted answer. To address this problem, we propose a novel read-then-verify system, which not only utilizes a neural reader to extract candidate answers and produce no-answer probabilities, but also leverages an answer verifier to decide whether the predicted answer is entailed by the input snippets. Moreover, we introduce two auxiliary losses to help the reader better handle answer extraction as well as no-answer detection, and investigate three different architectures for the answer verifier. Our experiments on the SQuAD 2.0 dataset show that our system achieves a score of 74.2 F1 on the test set, achieving state-of-the-art results at the time of submission (Aug. 28th, 2018).
arXiv commentTo appear at AAAI-19
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Teaching Machines to Read and Comprehend
- Enhanced LSTM for Natural Language Inference
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- A large annotated corpus for learning natural language inference
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
- Know What You Don't Know: Unanswerable Questions for SQuAD
- FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
- NewsQA: A Machine Comprehension Dataset
- Generating Wikipedia by Summarizing Long Sequences
- A Decomposable Attention Model for Natural Language Inference
- A Fast Unified Model for Parsing and Sentence Understanding
- Attention-Guided Answer Distillation for Machine Reading Comprehension
Cited by in corpus (10)
- Unified Language Model Pre-training for Natural Language Understanding and Generation
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering
- Introduction to Neural Network based Approaches for Question Answering over Knowledge Graphs
- U-Net: Machine Reading Comprehension with Unanswerable Questions
- SG-Net: Syntax-Guided Machine Reading Comprehension
- Stochastic Answer Networks for SQuAD 2.0
- Controlling Risk of Web Question Answering
- Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
- Answer Generation through Unified Memories over Multiple Passages