End-to-End Differentiable Proving

Abstract

We introduce neural networks for end-to-end differentiable proving of queries to knowledge bases by operating on dense vector representations of symbols. These neural networks are constructed recursively by taking inspiration from the backward chaining algorithm as used in Prolog. Specifically, we replace symbolic unification with a differentiable computation on vector representations of symbols using a radial basis function kernel, thereby combining symbolic reasoning with learning subsymbolic vector representations. By using gradient descent, the resulting neural network can be trained to infer facts from a given incomplete knowledge base. It learns to (i) place representations of similar symbols in close proximity in a vector space, (ii) make use of such similarities to prove queries, (iii) induce logical rules, and (iv) use provided and induced logical rules for multi-hop reasoning. We demonstrate that this architecture outperforms ComplEx, a state-of-the-art neural link prediction model, on three out of four benchmark knowledge bases while at the same time inducing interpretable function-free first-order logic rules.

NIPS 2017 camera-ready, NIPS 2017

References in corpus (11)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Complex Embeddings for Simple Link Prediction
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• Harnessing Deep Neural Networks with Logic Rules
• Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge
• Programming with a Differentiable Forth Interpreter
• Compositional Vector Space Models for Knowledge Base Completion
• TensorLog: A Differentiable Deductive Database
• HolStep: A Machine Learning Dataset for Higher-order Logic Theorem Proving
• Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies
• Separating Answers from Queries for Neural Reading Comprehension

Cited by in corpus (20)

• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• Learning to Plan Chemical Syntheses
• Knowledge Graphs
• TabFact: A Large-scale Dataset for Table-based Fact Verification
• DeepProbLog: Neural Probabilistic Logic Programming
• GamePad: A Learning Environment for Theorem Proving
• Multi-Hop Knowledge Graph Reasoning with Reward Shaping
• Logical Rule Induction and Theory Learning Using Neural Theorem Proving
• Towards Neural Theorem Proving at Scale
• Neural Guided Constraint Logic Programming for Program Synthesis
• Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks
• From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
• PAC-Reasoning in Relational Domains
• Combining Rules and Embeddings via Neuro-Symbolic AI for Knowledge Base Completion
• pix2rule: End-to-end Neuro-symbolic Rule Learning
• Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs
• Compositional Language Understanding with Text-based Relational Reasoning
• RuDaS: Synthetic Datasets for Rule Learning and Evaluation Tools
• RotLSTM: Rotating Memories in Recurrent Neural Networks
• Set Cross Entropy: Likelihood-based Permutation Invariant Loss Function for Probability Distributions