Differentiable Reasoning over a Virtual Knowledge Base

Abstract

We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB). In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus. At each step the module uses a combination of sparse-matrix TFIDF indices and a maximum inner product search (MIPS) on a special index of contextual representations of the mentions. This module is differentiable, so the full system can be trained end-to-end using gradient based methods, starting from natural language inputs. We also describe a pretraining scheme for the contextual representation encoder by generating hard negative examples using existing knowledge bases. We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%. On HotpotQA, DrKIT leads to a 10% improvement over a BERT-based re-ranking approach to retrieving the relevant passages required to answer a question. DrKIT is also very efficient, processing 10-100x more queries per second than existing multi-hop systems.

ICLR 2020

References in corpus (6)

• Bidirectional Attention Flow for Machine Comprehension
• Practical and Optimal LSH for Angular Distance
• What do you learn from context? Probing for sentence structure in contextualized word representations
• BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering
• A Discrete Hard EM Approach for Weakly Supervised Question Answering
• Neural Query Language: A Knowledge Base Query Language for Tensorflow

Cited by in corpus (19)

• Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering
• Modifying Memories in Transformer Models
• Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge
• Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
• Neuro-Symbolic Artificial Intelligence: Current Trends
• Answering Questions on COVID-19 in Real-Time
• Reasoning Over Virtual Knowledge Bases With Open Predicate Relations
• Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval
• Iterative Hierarchical Attention for Answering Complex Questions over Long Documents
• Open Question Answering over Tables and Text
• SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers
• Answering Open-Domain Questions of Varying Reasoning Steps from Text
• Answering Any-hop Open-domain Questions with Iterative Document Reranking
• Hopper: Multi-hop Transformer for Spatiotemporal Reasoning
• Adaptive Information Seeking for Open-Domain Question Answering
• Stronger Transformers for Neural Multi-Hop Question Generation
• Neural Unification for Logic Reasoning over Natural Language
• Relation-Guided Pre-Training for Open-Domain Question Answering
• Complex Knowledge Base Question Answering: A Survey