Neural Network Attributions: A Causal Perspective

Abstract

We propose a new attribution method for neural networks developed using first principles of causality (to the best of our knowledge, the first such). The neural network architecture is viewed as a Structural Causal Model, and a methodology to compute the causal effect of each feature on the output is presented. With reasonable assumptions on the causal structure of the input data, we propose algorithms to efficiently compute the causal effects, as well as scale the approach to data with large dimensionality. We also show how this method can be used for recurrent neural networks. We report experimental results on both simulated and real datasets showcasing the promise and usefulness of the proposed algorithm.

17 pages, 10 Figures. Accepted in the Proceedings of the 36th International Conference on Machine Learning (ICML2019). Modifications: Added github link to code and fixed a typo in Fig. 3

References in corpus (10)

• Semi-Supervised Learning with Deep Generative Models
• Understanding Neural Networks Through Deep Visualization
• SmoothGrad: removing noise by adding noise
• Grad-CAM: Why did you say that?
• Distilling a Neural Network Into a Soft Decision Tree
• The (Un)reliability of saliency methods
• Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions
• Inferring deterministic causal relations
• CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training
• The Do-Calculus Revisited