HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering

Abstract

Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers. We introduce HotpotQA, a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) we provide sentence-level supporting facts required for reasoning, allowing QA systems to reason with strong supervision and explain the predictions; (4) we offer a new type of factoid comparison questions to test QA systems' ability to extract relevant facts and perform necessary comparison. We show that HotpotQA is challenging for the latest QA systems, and the supporting facts enable models to improve performance and make explainable predictions.

arXiv commentEMNLP 2018 long paper. The first three authors contribute equally. Data, code, and blog posts available at https://hotpotqa.github.io/

References in corpus (5)

• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
• Know What You Don't Know: Unanswerable Questions for SQuAD
• DCN+: Mixed Objective and Deep Residual Coattention for Question Answering
• MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension

Cited by in corpus (60)

• CTRL: A Conditional Transformer Language Model for Controllable Generation
• BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
• Big Bird: Transformers for Longer Sequences
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension
• Autoregressive Entity Retrieval
• The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures
• Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
• Adversarial NLI: A New Benchmark for Natural Language Understanding
• Neural Module Networks for Reasoning over Text
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Towards Continual Knowledge Learning of Language Models
• CO-Search: COVID-19 Information Retrieval with Semantic Search, Question Answering, and Abstractive Summarization
• A Survey on Explainability in Machine Reading Comprehension
• ETC: Encoding Long and Structured Inputs in Transformers
• Cognitive Graph for Multi-Hop Reading Comprehension at Scale
• Review Conversational Reading Comprehension
• Conditional Self-Attention for Query-based Summarization
• Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
• ELI5: Long Form Question Answering
• MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension
• Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering
• ERNIE-Doc: A Retrospective Long-Document Modeling Transformer
• Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering
• Interactive Language Learning by Question Answering
• CAiRE-COVID: A Question Answering and Query-focused Multi-Document Summarization System for COVID-19 Scholarly Information Management
• Knowledge-Aided Open-Domain Question Answering
• Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting
• A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC
• Do Multi-hop Readers Dream of Reasoning Chains?
• Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA
• LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
• Break It Down: A Question Understanding Benchmark
• Can NLI Models Verify QA Systems' Predictions?
• A Self-supervised Method for Entity Alignment
• SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
• Semantic Graphs for Generating Deep Questions
• Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challenge
• Complementary Evidence Identification in Open-Domain Question Answering
• Multi-hop Reading Comprehension across Documents with Path-based Graph Convolutional Network
• VisBERT: Hidden-State Visualizations for Transformers
• Improving Cross-Lingual Reading Comprehension with Self-Training
• MeDiaQA: A Question Answering Dataset on Medical Dialogues
• A Study of the Tasks and Models in Machine Reading Comprehension
• Asking Complex Questions with Multi-hop Answer-focused Reasoning
• Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study
• Recent Advances in Automated Question Answering In Biomedical Domain
• Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
• A Multilingual Modeling Method for Span-Extraction Reading Comprehension
• Situation and Behavior Understanding by Trope Detection on Films
• Evaluating Document Coherence Modelling
• Understanding in Artificial Intelligence
• ComQA:Compositional Question Answering via Hierarchical Graph Neural Networks
• CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering
• Parameter-Efficient Neural Question Answering Models via Graph-Enriched Document Representations
• Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance
• Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?
• When to Fold'em: How to answer Unanswerable questions
• Fine-tuning Multi-hop Question Answering with Hierarchical Graph Network
• Better Understanding Hierarchical Visual Relationship for Image Caption