Calibrated Explanations: with Uncertainty Information and Counterfactuals

Abstract

While local explanations for AI models can offer insights into individual predictions, such as feature importance, they are plagued by issues like instability. The unreliability of feature weights, often skewed due to poorly calibrated ML models, deepens these challenges. Moreover, the critical aspect of feature importance uncertainty remains mostly unaddressed in Explainable AI (XAI). The novel feature importance explanation method presented in this paper, called Calibrated Explanations (CE), is designed to tackle these issues head-on. Built on the foundation of Venn-Abers, CE not only calibrates the underlying model but also delivers reliable feature importance explanations with an exact definition of the feature weights. CE goes beyond conventional solutions by addressing output uncertainty. It accomplishes this by providing uncertainty quantification for both feature weights and the model's probability estimates. Additionally, CE is model-agnostic, featuring easily comprehensible conditional rules and the ability to generate counterfactual explanations with embedded uncertainty quantification. Results from an evaluation with 25 benchmark datasets underscore the efficacy of CE, making it stand as a fast, reliable, stable, and robust solution.

arXiv comment19 pages, 6 figures, 3 tables, submitted to journal

References in corpus (14)

• A Unified Approach to Interpreting Model Predictions
• Towards A Rigorous Science of Interpretable Machine Learning
• Interpretable machine learning: definitions, methods, and applications
• Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations
• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems
• Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
• Model Agnostic Supervised Local Explanations
• Post-hoc explanation of black-box classifiers using confident itemsets
• S-LIME: Stabilized-LIME for Model Explanation
• OpenXAI: Towards a Transparent Evaluation of Model Explanations
• Getting a CLUE: A Method for Explaining Uncertainty Estimates
• A study of data and label shift in the LIME framework
• Large-scale probabilistic predictors with and without guarantees of validity

Cited by in corpus (3)

• Calibrating AI Models for Few-Shot Demodulation via Conformal Prediction
• CONFIDERAI: a novel CONFormal Interpretable-by-Design score function for Explainable and Reliable Artificial Intelligence
• The Impact of Machine Learning Uncertainty on the Robustness of Counterfactual Explanations