Discriminative Attribution from Counterfactuals

Abstract

We present a method for neural network interpretability by combining feature attribution with counterfactual explanations to generate attribution maps that highlight the most discriminative features between pairs of classes. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner, thus preventing potential observer bias. We evaluate the proposed method on three diverse datasets, including a challenging artificial dataset and real-world biological data. We show quantitatively and qualitatively that the highlighted features are substantially more discriminative than those extracted using conventional attribution methods and argue that this type of explanation is better suited for understanding fine grained class differences as learned by a deep neural network.

References in corpus (15)

• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• SmoothGrad: removing noise by adding noise
• How to Explain Individual Classification Decisions
• Model-Agnostic Interpretability of Machine Learning
• Sanity Checks for Saliency Maps
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• A Benchmark for Interpretability Methods in Deep Neural Networks
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Mode Regularized Generative Adversarial Networks
• Generative Counterfactual Introspection for Explainable Deep Learning