Quantifying the Intrinsic Usefulness of Attributional Explanations for Graph Neural Networks with Artificial Simulatability Studies

Abstract

Despite the increasing relevance of explainable AI, assessing the quality of explanations remains a challenging issue. Due to the high costs associated with human-subject experiments, various proxy metrics are often used to approximately quantify explanation quality. Generally, one possible interpretation of the quality of an explanation is its inherent value for teaching a related concept to a student. In this work, we extend artificial simulatability studies to the domain of graph neural networks. Instead of costly human trials, we use explanation-supervisable graph neural networks to perform simulatability studies to quantify the inherent usefulness of attributional graph explanations. We perform an extensive ablation study to investigate the conditions under which the proposed analyses are most meaningful. We additionally validate our methods applicability on real-world graph classification and regression datasets. We find that relevant explanations can significantly boost the sample efficiency of graph neural networks and analyze the robustness towards noise and bias in the explanations. We believe that the notion of usefulness obtained from our proposed simulatability analysis provides a dimension of explanation quality that is largely orthogonal to the common practice of faithfulness and has great potential to expand the toolbox of explanation quality assessments, specifically for graph explanations.

arXiv comment22 pages, accepted at xAI conference 2023 Portugal

References in corpus (11)

• On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
• Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoning
• Implementing graph neural networks with TensorFlow-Keras
• Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
• GRETEL: A unified framework for Graph Counterfactual Explanation Evaluation
• GCExplainer: Human-in-the-Loop Concept-based Explanations for Graph Neural Networks
• Learning to Scaffold: Optimizing Model Explanations for Teaching
• Encoding Concepts in Graph Neural Networks
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?
• PAGE: Prototype-Based Model-Level Explanations for Graph Neural Networks
• The Explanation Game: Towards Prediction Explainability through Sparse Communication