Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features

Abstract

Explainable Artificial Intelligence (XAI) plays a crucial role in fostering transparency and trust in AI systems, where traditional XAI approaches typically offer one level of abstraction for explanations, often in the form of heatmaps highlighting single or multiple input features. However, we ask whether abstract reasoning or problem-solving strategies of a model may also be relevant, as these align more closely with how humans approach solutions to problems. We propose a framework, called Symbolic XAI, that attributes relevance to symbolic queries expressing logical relationships between input features, thereby capturing the abstract reasoning behind a model's predictions. The methodology is built upon a simple yet general multi-order decomposition of model predictions. This decomposition can be specified using higher-order propagation-based relevance methods, such as GNN-LRP, or perturbation-based explanation methods commonly used in XAI. The effectiveness of our framework is demonstrated in the domains of natural language processing (NLP), vision, and quantum chemistry (QC), where abstract symbolic domain knowledge is abundant and of significant interest to users. The Symbolic XAI framework provides an understanding of the model's decision-making process that is both flexible for customization by the user and human-readable through logical formulas.

References in corpus (38)

• Geometric deep learning: going beyond Euclidean data
• Methods for Interpreting and Understanding Deep Neural Networks
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• SchNet - a deep learning architecture for molecules and materials
• On the Opportunities and Risks of Foundation Models
• E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
• PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments and Partial Charges
• How to Explain Individual Classification Decisions
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
• SchNetPack: A Deep Learning Toolbox For Atomistic Systems
• SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects
• Managing extreme AI risks amid rapid progress
• Equivariant message passing for the prediction of tensorial properties and molecular spectra
• Higher-Order Explanations of Graph Neural Networks via Relevant Walks
• Deep Potential: a general representation of a many-body potential energy surface
• Parameterized Explainer for Graph Neural Network
• From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
• Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations
• Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry
• SchNetPack 2.0: A neural network toolbox for atomistic machine learning
• Logic Explained Networks
• Rule Extraction Algorithm for Deep Neural Networks: A Review
• Building and Interpreting Deep Similarity Models
• Getting aligned on representational alignment
• Quantum chemical roots of machine-learning molecular similarity descriptors
• Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces
• So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems
• PredDiff: Explanations and Interactions from Conditional Expectations
• Insightful analysis of historical sources at scales beyond human capabilities using unsupervised Machine Learning and XAI
• Peering inside the black box: Learning the relevance of many-body functions in Neural Network potentials
• Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks
• xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology
• MambaLRP: Explaining Selective State Space Sequence Models

Cited by in corpus (1)

• Analyzing Atomic Interactions in Molecules as Learned by Neural Networks