Consistent Counterfactuals for Deep Models
arXiv:2110.03109
Abstract
Counterfactual examples are one of the most commonly-cited methods for explaining the predictions of machine learning models in key areas such as finance and medical diagnosis. Counterfactuals are often discussed under the assumption that the model on which they will be used is static, but in deployment models may be periodically retrained or fine-tuned. This paper studies the consistency of model prediction on counterfactual examples in deep networks under small changes to initial training conditions, such as weight initialization and leave-one-out variations in data, as often occurs during model deployment. We demonstrate experimentally that counterfactual examples for deep models are often inconsistent across such small changes, and that increasing the cost of the counterfactual, a stability-enhancing mitigation suggested by prior work in the context of simpler models, is not a reliable heuristic in deep networks. Rather, our analysis shows that a model's local Lipschitz continuity around the counterfactual is key to its consistency across related models. To this end, we propose Stable Neighbor Search as a way to generate more consistent counterfactual explanations, and illustrate the effectiveness of this approach on several benchmark datasets.
References in corpus (18)
- A Survey on Deep Learning in Medical Image Analysis
- Explaining and Harnessing Adversarial Examples
- Intriguing properties of neural networks
- Axiomatic Attribution for Deep Networks
- Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
- Actionable Recourse in Linear Classification
- Underspecification Presents Challenges for Credibility in Modern Machine Learning
- The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons
- Local Rule-Based Explanations of Black Box Decision Systems
- Learning Model-Agnostic Counterfactual Explanations for Tabular Data
- Explanations can be manipulated and geometry is to blame
- Deep ReLU Networks Have Surprisingly Few Activation Patterns
- Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
- Algorithmic recourse under imperfect causal knowledge: a probabilistic approach
- Provable Robustness of ReLU networks via Maximization of Linear Regions
- Predictive Multiplicity in Classification
- Issues with post-hoc counterfactual explanations: a discussion
- Semantics and explanation: why counterfactual explanations produce adversarial examples in deep neural networks