Multi-hop Question Answering via Reasoning Chains

Abstract

Multi-hop question answering requires models to gather information from different parts of a text to answer a question. Most current approaches learn to address this task in an end-to-end way with neural networks, without maintaining an explicit representation of the reasoning process. We propose a method to extract a discrete reasoning chain over the text, which consists of a series of sentences leading to the answer. We then feed the extracted chains to a BERT-based QA model to do final answer prediction. Critically, we do not rely on gold annotated chains or "supporting facts:" at training time, we derive pseudogold reasoning chains using heuristics based on named entity recognition and coreference resolution. Nor do we rely on these annotations at test time, as our model learns to extract chains from raw text alone. We test our approach on two recently proposed large multi-hop question answering datasets: WikiHop and HotpotQA, and achieve state-of-art performance on WikiHop and strong performance on HotpotQA. Our analysis shows the properties of chains that are crucial for high performance: in particular, modeling extraction sequentially is important, as is dealing with each candidate sentence in a context-aware way. Furthermore, human evaluation shows that our extracted chains allow humans to give answers with high confidence, indicating that these are a strong intermediate abstraction for this task.

References in corpus (9)

• Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
• Coarse-grain Fine-grain Coattention Network for Multi-evidence Question Answering
• Hierarchical Graph Network for Multi-hop Question Answering
• Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs
• Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering
• Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documents
• Answering Complex Open-domain Questions Through Iterative Query Generation
• Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension
• Multi-Hop Paragraph Retrieval for Open-Domain Question Answering

Cited by in corpus (19)

• Longformer: The Long-Document Transformer
• Big Bird: Transformers for Longer Sequences
• Rethinking Search: Making Domain Experts out of Dilettantes
• A Neural Entity Coreference Resolution Review
• A Survey on Explainability in Machine Reading Comprehension
• Hierarchical Graph Network for Multi-hop Question Answering
• Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2
• Multi-Step Inference for Reasoning Over Paragraphs
• Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy
• Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering
• Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference
• Natural Language Inference in Context -- Investigating Contextual Reasoning over Long Texts
• Learning to Recover Reasoning Chains for Multi-Hop Question Answering via Cooperative Games
• Hopper: Multi-hop Transformer for Spatiotemporal Reasoning
• Stronger Transformers for Neural Multi-Hop Question Generation
• Explaining the Road Not Taken
• Relation/Entity-Centric Reading Comprehension
• SRLGRN: Semantic Role Labeling Graph Reasoning Network
• Generative Context Pair Selection for Multi-hop Question Answering