Incorporating Symbolic Domain Knowledge into Graph Neural Networks

Abstract

Our interest is in scientific problems with the following characteristics: (1) Data are naturally represented as graphs; (2) The amount of data available is typically small; and (3) There is significant domain-knowledge, usually expressed in some symbolic form. These kinds of problems have been addressed effectively in the past by Inductive Logic Programming (ILP), by virtue of 2 important characteristics: (a) The use of a representation language that easily captures the relation encoded in graph-structured data, and (b) The inclusion of prior information encoded as domain-specific relations, that can alleviate problems of data scarcity, and construct new relations. Recent advances have seen the emergence of deep neural networks specifically developed for graph-structured data (Graph-based Neural Networks, or GNNs). While GNNs have been shown to be able to handle graph-structured data, less has been done to investigate the inclusion of domain-knowledge. Here we investigate this aspect of GNNs empirically by employing an operation we term "vertex-enrichment" and denote the corresponding GNNs as "VEGNNs". Using over 70 real-world datasets and substantial amounts of symbolic domain-knowledge, we examine the result of vertex-enrichment across 5 different variants of GNNs. Our results provide support for the following: (a) Inclusion of domain-knowledge by vertex-enrichment can significantly improve the performance of a GNN. That is, the performance VEGNNs is significantly better than GNNs across all GNN variants; (b) The inclusion of domain-specific relations constructed using ILP improves the performance of VEGNNs, across all GNN variants. Taken together, the results provide evidence that it is possible to incorporate symbolic domain knowledge into a GNN, and that ILP can play an important role in providing high-level relationships that are not easily discovered by a GNN.

arXiv commentAccepted in Machine Learning Journal (MLJ)

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A Comprehensive Survey on Graph Neural Networks
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Graph Neural Networks: A Review of Methods and Applications
• Fast Graph Representation Learning with PyTorch Geometric
• Graph Neural Networks with convolutional ARMA filters
• Self-Attention Graph Pooling
• Neural-Symbolic Learning and Reasoning: A Survey and Interpretation
• HyperGCN: A New Method of Training Graph Convolutional Networks on Hypergraphs
• Towards Sparse Hierarchical Graph Classifiers
• A Survey on The Expressive Power of Graph Neural Networks
• Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning

Cited by in corpus (5)

• A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks
• Incorporating Domain Knowledge into Deep Neural Networks
• Inclusion of Domain-Knowledge into GNNs using Mode-Directed Inverse Entailment
• Superpixel-based Knowledge Infusion in Deep Neural Networks for Image Classification
• Composition of Relational Features with an Application to Explaining Black-Box Predictors