Beyond Graph Neural Networks with Lifted Relational Neural Networks

Abstract

We demonstrate a declarative differentiable programming framework based on the language of Lifted Relational Neural Networks, where small parameterized logic programs are used to encode relational learning scenarios. When presented with relational data, such as various forms of graphs, the program interpreter dynamically unfolds differentiable computational graphs to be used for the program parameter optimization by standard means. Following from the used declarative Datalog abstraction, this results into compact and elegant learning programs, in contrast with the existing procedural approaches operating directly on the computational graph level. We illustrate how this idea can be used for an efficient encoding of a diverse range of existing advanced neural architectures, with a particular focus on Graph Neural Networks (GNNs). Additionally, we show how the contemporary GNN models can be easily extended towards higher relational expressiveness. In the experiments, we demonstrate correctness and computation efficiency through comparison against specialized GNN deep learning frameworks, while shedding some light on the learning performance of existing GNN models.

arXiv commentSubmitted to MLJ's Special Track on Learning and Reasoning (May 15th 2020 cut-off) http://lr2020.iit.demokritos.gr/

References in corpus (26)

• A Comprehensive Survey on Graph Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• A Critical Review of Recurrent Neural Networks for Sequence Learning
• Graph Neural Networks: A Review of Methods and Applications
• Fast Graph Representation Learning with PyTorch Geometric
• Representation Learning on Graphs with Jumping Knowledge Networks
• Learning Convolutional Neural Networks for Graphs
• Revisiting Small Batch Training for Deep Neural Networks
• DyNet: The Dynamic Neural Network Toolkit
• Benchmarking Graph Neural Networks
• Differentiable Learning of Logical Rules for Knowledge Base Reasoning
• Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge
• Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective
• Supervised Community Detection with Line Graph Neural Networks
• DeepProbLog: Neural Probabilistic Logic Programming
• Heterogeneous Information Network Embedding for Meta Path based Proximity
• Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning
• TensorLog: A Differentiable Deductive Database
• Ontology Reasoning with Deep Neural Networks
• Dimensions of Neural-symbolic Integration - A Structured Survey
• Towards Neural Theorem Proving at Scale
• Lossless Compression of Structured Convolutional Models via Lifting
• Learning with Molecules beyond Graph Neural Networks

Cited by in corpus (4)

• A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks
• Incorporating Domain Knowledge into Deep Neural Networks
• Inclusion of Domain-Knowledge into GNNs using Mode-Directed Inverse Entailment
• Learning with Molecules beyond Graph Neural Networks