Collaborative Policy Learning for Open Knowledge Graph Reasoning

Abstract

In recent years, there has been a surge of interests in interpretable graph reasoning methods. However, these models often suffer from limited performance when working on sparse and incomplete graphs, due to the lack of evidential paths that can reach target entities. Here we study open knowledge graph reasoning---a task that aims to reason for missing facts over a graph augmented by a background text corpus. A key challenge of the task is to filter out "irrelevant" facts extracted from corpus, in order to maintain an effective search space during path inference. We propose a novel reinforcement learning framework to train two collaborative agents jointly, i.e., a multi-hop graph reasoner and a fact extractor. The fact extraction agent generates fact triples from corpora to enrich the graph on the fly; while the reasoning agent provides feedback to the fact extractor and guides it towards promoting facts that are helpful for the interpretable reasoning. Experiments on two public datasets demonstrate the effectiveness of the proposed approach. Source code and datasets used in this paper can be downloaded at https://github.com/shanzhenren/CPL

References in corpus (7)

• Playing Atari with Deep Reinforcement Learning
• RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
• Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
• DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning
• Joint Representation Learning of Text and Knowledge for Knowledge Graph Completion
• Multi-Hop Knowledge Graph Reasoning with Reward Shaping
• Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text

Cited by in corpus (2)

• Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering
• Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph