Reliable Post hoc Explanations: Modeling Uncertainty in Explainability
arXiv:2008.05030
Abstract
As black box explanations are increasingly being employed to establish model credibility in high-stakes settings, it is important to ensure that these explanations are accurate and reliable. However, prior work demonstrates that explanations generated by state-of-the-art techniques are inconsistent, unstable, and provide very little insight into their correctness and reliability. In addition, these methods are also computationally inefficient, and require significant hyper-parameter tuning. In this paper, we address the aforementioned challenges by developing a novel Bayesian framework for generating local explanations along with their associated uncertainty. We instantiate this framework to obtain Bayesian versions of LIME and KernelSHAP which output credible intervals for the feature importances, capturing the associated uncertainty. The resulting explanations not only enable us to make concrete inferences about their quality (e.g., there is a 95% chance that the feature importance lies within the given range), but are also highly consistent and stable. We carry out a detailed theoretical analysis that leverages the aforementioned uncertainty to estimate how many perturbations to sample, and how to sample for faster convergence. This work makes the first attempt at addressing several critical issues with popular explanation methods in one shot, thereby generating consistent, stable, and reliable explanations with guarantees in a computationally efficient manner. Experimental evaluation with multiple real world datasets and user studies demonstrate that the efficacy of the proposed framework.
References in corpus (19)
- Axiomatic Attribution for Deep Networks
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Understanding Black-box Predictions via Influence Functions
- SmoothGrad: removing noise by adding noise
- Model-Agnostic Interpretability of Machine Learning
- Sanity Checks for Saliency Maps
- Actionable Recourse in Linear Classification
- Explanations can be manipulated and geometry is to blame
- Model Agnostic Supervised Local Explanations
- Fooling Neural Network Interpretations via Adversarial Model Manipulation
- Interpretability via Model Extraction
- CXPlain: Causal Explanations for Model Interpretation under Uncertainty
- "Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
- Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods
- Explaining Deep Learning Models - A Bayesian Non-parametric Approach
- BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations
- Supersparse Linear Integer Models for Interpretable Classification
- Certifiably Robust Interpretation in Deep Learning
- Mapping chemical performance on molecular structures using locally interpretable explanations