Reasoning Over Virtual Knowledge Bases With Open Predicate Relations
arXiv:2102.07043
Abstract
We present the Open Predicate Query Language (OPQL); a method for constructing a virtual KB (VKB) trained entirely from text. Large Knowledge Bases (KBs) are indispensable for a wide-range of industry applications such as question answering and recommendation. Typically, KBs encode world knowledge in a structured, readily accessible form derived from laborious human annotation efforts. Unfortunately, while they are extremely high precision, KBs are inevitably highly incomplete and automated methods for enriching them are far too inaccurate. Instead, OPQL constructs a VKB by encoding and indexing a set of relation mentions in a way that naturally enables reasoning and can be trained without any structured supervision. We demonstrate that OPQL outperforms prior VKB methods on two different KB reasoning tasks and, additionally, can be used as an external memory integrated into a language model (OPQL-LM) leading to improvements on two open-domain question answering tasks.
arXiv commentAccepted at the 38th International Conference on Machine Learning, PMLR 139, 2021
References in corpus (11)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- REALM: Retrieval-Augmented Language Model Pre-Training
- Neural Natural Language Inference Models Enhanced with External Knowledge
- Variational Reasoning for Question Answering with Knowledge Graph
- A Neural Knowledge Language Model
- Dynamic Integration of Background Knowledge in Neural NLU Systems
- Differentiable Reasoning over a Virtual Knowledge Base
- Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge
- Learning Cross-Context Entity Representations from Text
- Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base