Higher-Order Explanations of Graph Neural Networks via Relevant Walks

Abstract

Graph Neural Networks (GNNs) are a popular approach for predicting graph structured data. As GNNs tightly entangle the input graph into the neural network structure, common explainable AI approaches are not applicable. To a large extent, GNNs have remained black-boxes for the user so far. In this paper, we show that GNNs can in fact be naturally explained using higher-order expansions, i.e. by identifying groups of edges that jointly contribute to the prediction. Practically, we find that such explanations can be extracted using a nested attribution scheme, where existing techniques such as layer-wise relevance propagation (LRP) can be applied at each step. The output is a collection of walks into the input graph that are relevant for the prediction. Our novel explanation method, which we denote by GNN-LRP, is applicable to a broad range of graph neural networks and lets us extract practically relevant insights on sentiment analysis of text data, structure-property relationships in quantum chemistry, and image classification.

arXiv comment14 pages + 6 pages supplement

References in corpus (23)

• Statistical mechanics of complex networks
• A Comprehensive Survey on Graph Neural Networks
• Geometric deep learning: going beyond Euclidean data
• Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting
• Methods for Interpreting and Understanding Deep Neural Networks
• SchNet - a deep learning architecture for molecules and materials
• Machine Learning Force Fields
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Modeling polypharmacy side effects with graph convolutional networks
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
• Machine learning for molecular simulation
• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
• SchNetPack: A Deep Learning Toolbox For Atomistic Systems
• XGNN: Towards Model-Level Explanations of Graph Neural Networks
• Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
• Explainability in Graph Neural Networks: A Taxonomic Survey
• Explaining and Interpreting LSTMs
• Building and Interpreting Deep Similarity Models
• GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks

Cited by in corpus (51)

• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
• Neural Network Potentials for Chemistry: Concepts, Applications and Prospects
• From Clustering to Cluster Explanations via Neural Networks
• Toward Explainable AI for Regression Models
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• On Explainability of Graph Neural Networks via Subgraph Explorations
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Reinforced Causal Explainer for Graph Neural Networks
• DIG: A Turnkey Library for Diving into Graph Deep Learning Research
• Explaining the Explainers in Graph Neural Networks: a Comparative Study
• Graph-guided random forest for gene set selection
• Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking
• Higher-order Clustering and Pooling for Graph Neural Networks
• The Clever Hans Effect in Unsupervised Learning
• Deeper Insights into Deep Graph Convolutional Networks: Stability and Generalization
• Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanation
• ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splitting
• Explanation-Guided Training for Cross-Domain Few-Shot Classification
• GNNShap: Scalable and Accurate GNN Explanation using Shapley Values
• A Detailed Study of Interpretability of Deep Neural Network based Top Taggers
• Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces
• Analyzing Atomic Interactions in Molecules as Learned by Neural Networks
• Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph
• Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
• Decomposing Chemical Space: Applications to the Machine Learning of Atomic Energies
• PredDiff: Explanations and Interactions from Conditional Expectations
• Explaining Predictive Uncertainty by Exposing Second-Order Effects
• Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features
• FAIR AI Models in High Energy Physics
• Beyond Fidelity: Explaining Vulnerability Localization of Learning-based Detectors
• Discovering robust biomarkers of psychiatric disorders from resting-state functional MRI via graph neural networks: A systematic review
• Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI
• Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks
• TayFCS: Towards Light Feature Combination Selection for Deep Recommender Systems
• Globally Interpretable Graph Learning via Distribution Matching
• KANDINSKYPatterns -- An experimental exploration environment for Pattern Analysis and Machine Intelligence
• Generating Post-hoc Explanations for Skip-gram-based Node Embeddings by Identifying Important Nodes with Bridgeness
• An Empirical Study: Extensive Deep Temporal Point Process
• Explaining GNN Explanations with Edge Gradients
• Reliable Graph Neural Network Explanations Through Adversarial Training
• Graph Denoising with Framelet Regularizer
• Sparse Explanations of Neural Networks Using Pruned Layer-Wise Relevance Propagation
• The GECo algorithm for Graph Neural Networks Explanation
• Towards Desiderata-Driven Design of Visual Counterfactual Explainers
• Explaining GNN over Evolving Graphs using Information Flow
• Fast and Accurate Explanations of Distance-Based Classifiers by Uncovering Latent Explanatory Structures
• Split and Expand: An inference-time improvement for Weakly Supervised Cell Instance Segmentation
• SEEN: Sharpening Explanations for Graph Neural Networks using Explanations from Neighborhoods
• Explaining machine-learned particle-flow reconstruction
• MBExplainer: Multilevel bandit-based explanations for downstream models with augmented graph embeddings