Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking

Abstract

Graph neural networks (GNNs) have become a popular approach to integrating structural inductive biases into NLP models. However, there has been little work on interpreting them, and specifically on understanding which parts of the graphs (e.g. syntactic trees or co-reference structures) contribute to a prediction. In this work, we introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges. Given a trained GNN model, we learn a simple classifier that, for every edge in every layer, predicts if that edge can be dropped. We demonstrate that such a classifier can be trained in a fully differentiable fashion, employing stochastic gates and encouraging sparsity through the expected $L_0$ norm. We use our technique as an attribution method to analyze GNN models for two tasks -- question answering and semantic role labeling -- providing insights into the information flow in these models. We show that we can drop a large proportion of edges without deteriorating the performance of the model, while we can analyse the remaining edges for interpreting model predictions.

References in corpus (24)

• Auto-Encoding Variational Bayes
• Categorical Reparameterization with Gumbel-Softmax
• Axiomatic Attribution for Deep Networks
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• Learning Important Features Through Propagating Activation Differences
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• How to Explain Individual Classification Decisions
• Model-Agnostic Interpretability of Machine Learning
• Attention is not Explanation
• Understanding Neural Networks through Representation Erasure
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Assessing BERT's Syntactic Abilities
• Higher-Order Explanations of Graph Neural Networks via Relevant Walks
• Pathologies of Neural Models Make Interpretations Difficult
• Parameterized Explainer for Graph Neural Network
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
• Structured Neural Summarization
• Hierarchical interpretations for neural network predictions
• Explainability Techniques for Graph Convolutional Networks
• Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering
• Restricting the Flow: Information Bottlenecks for Attribution
• A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
• Abusive Language Detection with Graph Convolutional Networks
• Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing

Cited by in corpus (9)

• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• A Meta-Learning Approach for Training Explainable Graph Neural Networks
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• BrainNNExplainer: An Interpretable Graph Neural Network Framework for Brain Network based Disease Analysis
• Mediators in Determining what Processing BERT Performs First
• Generating Post-hoc Explanations for Skip-gram-based Node Embeddings by Identifying Important Nodes with Bridgeness
• Towards Automated Evaluation of Explanations in Graph Neural Networks
• Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis
• Reimagining GNN Explanations with ideas from Tabular Data