Neural Network Attributions: A Causal Perspective

Abstract

We propose a new attribution method for neural networks developed using first principles of causality (to the best of our knowledge, the first such). The neural network architecture is viewed as a Structural Causal Model, and a methodology to compute the causal effect of each feature on the output is presented. With reasonable assumptions on the causal structure of the input data, we propose algorithms to efficiently compute the causal effects, as well as scale the approach to data with large dimensionality. We also show how this method can be used for recurrent neural networks. We report experimental results on both simulated and real datasets showcasing the promise and usefulness of the proposed algorithm.

17 pages, 10 Figures. Accepted in the Proceedings of the 36th International Conference on Machine Learning (ICML2019). Modifications: Added github link to code and fixed a typo in Fig. 3

References in corpus (15)

• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• Semi-Supervised Learning with Deep Generative Models
• Understanding Neural Networks Through Deep Visualization
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• SmoothGrad: removing noise by adding noise
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Grad-CAM: Why did you say that?
• Distilling a Neural Network Into a Soft Decision Tree
• The (Un)reliability of saliency methods
• Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
• Inferring deterministic causal relations
• CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training
• The Do-Calculus Revisited

Cited by in corpus (5)

• Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
• Feature relevance quantification in explainable AI: A causal problem
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation
• Disentangling causal effects for hierarchical reinforcement learning