Efficient and Robust Question Answering from Minimal Context over Documents
arXiv:1805.08092
Abstract
Neural models for question answering (QA) over documents have achieved significant performance improvements. Although effective, these models do not scale to large corpora due to their complex modeling of interactions between the document and the question. Moreover, recent work has shown that such models are sensitive to adversarial inputs. In this paper, we study the minimal context required to answer the question, and find that most questions in existing datasets can be answered with a small set of sentences. Inspired by this observation, we propose a simple sentence selector to select the minimal set of sentences to feed into the QA model. Our overall system achieves significant reductions in training (up to 15 times) and inference times (up to 13 times), with accuracy comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results and analyses show that our approach is more robust to adversarial inputs.
Published as a conference paper at ACL 2018 (long paper)
References in corpus (11)
- Adversarial Examples for Evaluating Reading Comprehension Systems
- Learned in Translation: Contextualized Word Vectors
- Quasar: Datasets for Question Answering by Search and Reading
- Learning Recurrent Span Representations for Extractive Question Answering
- R: Reinforced Reader-Ranker for Open-Domain Question Answering
- DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
- NewsQA: A Machine Comprehension Dataset
- MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
- Dynamic Integration of Background Knowledge in Neural NLU Systems
- Reinforced Mnemonic Reader for Machine Reading Comprehension
- Large-scale Cloze Test Dataset Created by Teachers
Cited by in corpus (13)
- Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
- Unsupervised Question Answering by Cloze Translation
- Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue
- Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering
- A Survey on Explainability in Machine Reading Comprehension
- Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives
- Knowledge-Aided Open-Domain Question Answering
- EfficientQA : a RoBERTa Based Phrase-Indexed Question-Answering System
- ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
- SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
- Complementary Evidence Identification in Open-Domain Question Answering
- MIX : a Multi-task Learning Approach to Solve Open-Domain Question Answering
- Machine Reading Comprehension using Case-based Reasoning