Improving Question Answering with External Knowledge

Abstract

We focus on multiple-choice question answering (QA) tasks in subject areas such as science, where we require both broad background knowledge and the facts from the given subject-area reference corpus. In this work, we explore simple yet effective methods for exploiting two sources of external knowledge for subject-area QA. The first enriches the original subject-area reference corpus with relevant text snippets extracted from an open-domain resource (i.e., Wikipedia) that cover potentially ambiguous concepts in the question and answer options. As in other QA research, the second method simply increases the amount of training data by appending additional in-domain subject-area instances. Experiments on three challenging multiple-choice science QA tasks (i.e., ARC-Easy, ARC-Challenge, and OpenBookQA) demonstrate the effectiveness of our methods: in comparison to the previous state-of-the-art, we obtain absolute gains in accuracy of up to 8.1%, 13.0%, and 12.8%, respectively. While we observe consistent gains when we introduce knowledge from Wikipedia, we find that employing additional QA training instances is not uniformly helpful: performance degrades when the added instances exhibit a higher level of difficulty than the original training data. As one of the first studies on exploiting unstructured external knowledge for subject-area QA, we hope our methods, observations, and discussion of the exposed limitations may shed light on further developments in the area.

Accepted to MRQA (2019)

References in corpus (12)

• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• End-to-End Open-Domain Question Answering with BERTserini
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• Quasar: Datasets for Question Answering by Search and Reading
• A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
• A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories
• R${}^3$: Reinforced Reader-Ranker for Open-Domain Question Answering
• Question Answering as Global Reasoning over Semantic Abstractions
• KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings
• Answering Science Exam Questions Using Query Rewriting with Background Knowledge
• Learning Distributed Representations of Texts and Entities from Knowledge Base

Cited by in corpus (8)

• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
• Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering
• Multi-class Hierarchical Question Classification for Multiple Choice Science Exams
• An MRC Framework for Semantic Role Labeling
• SirenLess: reveal the intention behind news
• Answering questions by learning to rank -- Learning to rank by answering questions
• Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System