Inclusion of Domain-Knowledge into GNNs using Mode-Directed Inverse Entailment

Abstract

We present a general technique for constructing Graph Neural Networks (GNNs) capable of using multi-relational domain knowledge. The technique is based on mode-directed inverse entailment (MDIE) developed in Inductive Logic Programming (ILP). Given a data instance $e$ and background knowledge $B$, MDIE identifies a most-specific logical formula ${\perp }_B(e)$ that contains all the relational information in $B$ that is related to $e$. We represent ${\perp }_B(e)$ by a "bottom-graph" that can be converted into a form suitable for GNN implementations. This transformation allows a principled way of incorporating generic background knowledge into GNNs: we use the term `BotGNN' for this form of graph neural networks. For several GNN variants, using real-world datasets with substantial background knowledge, we show that BotGNNs perform significantly better than both GNNs without background knowledge and a recently proposed simplified technique for including domain knowledge into GNNs. We also provide experimental evidence comparing BotGNNs favourably to multi-layer perceptrons (MLPs) that use features representing a "propositionalised" form of the background knowledge; and BotGNNs to a standard ILP based on the use of most-specific clauses. Taken together, these results point to BotGNNs as capable of combining the computational efficacy of GNNs with the representational versatility of ILP.

arXiv commentRevised version; submitted to Machine Learning Journal (MLJ)

References in corpus (18)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A Comprehensive Survey on Graph Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Inductive Representation Learning on Large Graphs
• Neural Message Passing for Quantum Chemistry
• Fast Graph Representation Learning with PyTorch Geometric
• Knowledge Graph Convolutional Networks for Recommender Systems
• Graph Neural Networks with convolutional ARMA filters
• Self-Attention Graph Pooling
• Towards Sparse Hierarchical Graph Classifiers
• Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels
• Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective
• Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning
• A Semantic Loss Function for Deep Learning with Symbolic Knowledge
• kLog: A Language for Logical and Relational Learning with Kernels
• Incorporating Symbolic Domain Knowledge into Graph Neural Networks
• Beyond Graph Neural Networks with Lifted Relational Neural Networks

Cited by in corpus (3)

• A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks
• Incorporating Domain Knowledge into Deep Neural Networks
• Composition of Relational Features with an Application to Explaining Black-Box Predictors