pix2rule: End-to-end Neuro-symbolic Rule Learning

Abstract

Humans have the ability to seamlessly combine low-level visual input with high-level symbolic reasoning often in the form of recognising objects, learning relations between them and applying rules. Neuro-symbolic systems aim to bring a unifying approach to connectionist and logic-based principles for visual processing and abstract reasoning respectively. This paper presents a complete neuro-symbolic method for processing images into objects, learning relations and logical rules in an end-to-end fashion. The main contribution is a differentiable layer in a deep learning architecture from which symbolic relations and rules can be extracted by pruning and thresholding. We evaluate our model using two datasets: subgraph isomorphism task for symbolic rule learning and an image classification domain with compound relations for learning objects, relations and rules. We demonstrate that our model scales beyond state-of-the-art symbolic learners and outperforms deep relational neural network architectures.

IJCLR-NeSy, 41 pages. Minor correction to Lukasiewicz logic

References in corpus (18)

• Categorical Reparameterization with Gumbel-Softmax
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding
• Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
• Graph Matching Networks for Learning the Similarity of Graph Structured Objects
• MONet: Unsupervised Scene Decomposition and Representation
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
• End-to-End Differentiable Proving
• DeepProbLog: Neural Probabilistic Logic Programming
• Logical Neural Networks
• TensorLog: A Differentiable Deductive Database
• Learning Physical Graph Representations from Visual Scenes
• Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary
• An Explicitly Relational Neural Network Architecture
• Unsupervised Grounding of Plannable First-Order Logic Representation from Images
• Learning Algorithms via Neural Logic Networks
• NSL: Hybrid Interpretable Learning From Noisy Raw Data
• Differentiable Inductive Logic Programming for Structured Examples