Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions

Abstract

Whilst an abundance of techniques have recently been proposed to generate counterfactual explanations for the predictions of opaque black-box systems, markedly less attention has been paid to exploring the uncertainty of these generated explanations. This becomes a critical issue in high-stakes scenarios, where uncertain and misleading explanations could have dire consequences (e.g., medical diagnosis and treatment planning). Moreover, it is often difficult to determine if the generated explanations are well grounded in the training data and sensitive to distributional shifts. This paper proposes several practical solutions that can be leveraged to solve these problems by establishing novel connections with other research works in explainability (e.g., trust scores) and uncertainty estimation (e.g., Monte Carlo Dropout). Two experiments demonstrate the utility of our proposed solutions.

References in corpus (15)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Actionable Recourse in Linear Classification
• Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers
• Reliable Decision Support using Counterfactual Models
• How model accuracy and explanation fidelity influence user trust
• Towards Robust and Reliable Algorithmic Recourse
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
• Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
• When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making
• To Trust or Not to Trust a Regressor: Estimating and Explaining Trustworthiness of Regression Predictions
• On Generating Plausible Counterfactual and Semi-Factual Explanations for Deep Learning
• Ordered Counterfactual Explanation by Mixed-Integer Linear Optimization