On the (In)fidelity and Sensitivity for Explanations

Abstract

We consider objective evaluation measures of saliency explanations for complex black-box machine learning models. We propose simple robust variants of two notions that have been considered in recent literature: (in)fidelity, and sensitivity. We analyze optimal explanations with respect to both these measures, and while the optimal explanation for sensitivity is a vacuous constant explanation, the optimal explanation for infidelity is a novel combination of two popular explanation methods. By varying the perturbation distribution that defines infidelity, we obtain novel explanations by optimizing infidelity, which we show to out-perform existing explanations in both quantitative and qualitative measurements. Another salient question given these measures is how to modify any given explanation to have better values with respect to these measures. We propose a simple modification based on lowering sensitivity, and moreover show that when done appropriately, we could simultaneously improve both sensitivity as well as fidelity.

NeurIPS 2019 camera ready, previous version on Arxiv: "How Sensitive are Sensitivity-Based Explanations"

References in corpus (20)

• Axiomatic Attribution for Deep Networks
• Methods for Interpreting and Understanding Deep Neural Networks
• Learning Important Features Through Propagating Activation Differences
• Understanding Black-box Predictions via Influence Functions
• SmoothGrad: removing noise by adding noise
• How to Explain Individual Classification Decisions
• GNNExplainer: Generating Explanations for Graph Neural Networks
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• Sanity Checks for Saliency Maps
• Certified Defenses against Adversarial Examples
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• A Benchmark for Interpretability Methods in Deep Neural Networks
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Model Agnostic Supervised Local Explanations
• Explaining Image Classifiers by Counterfactual Generation
• Representer Point Selection for Explaining Deep Neural Networks
• Interpreting Black Box Predictions using Fisher Kernels
• Towards Robust, Locally Linear Deep Networks

Cited by in corpus (13)

• Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing
• Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks
• Impossibility Theorems for Feature Attribution
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• Do Not Trust Additive Explanations
• Adversarial Infidelity Learning for Model Interpretation
• Quantitative Evaluations on Saliency Methods: An Experimental Study
• Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks
• On Locality of Local Explanation Models
• Quantifying Explainability in NLP and Analyzing Algorithms for Performance-Explainability Tradeoff
• Applicability Evaluation of Selected xAI Methods for Machine Learning Algorithms for Signal Parameters Extraction
• EDDA: Explanation-driven Data Augmentation to Improve Explanation Faithfulness
• Balancing Robustness and Sensitivity using Feature Contrastive Learning