CXPlain: Causal Explanations for Model Interpretation under Uncertainty

Abstract

Feature importance estimates that inform users about the degree to which given inputs influence the output of a predictive model are crucial for understanding, validating, and interpreting machine-learning models. However, providing fast and accurate estimates of feature importance for high-dimensional data, and quantifying the uncertainty of such estimates remain open challenges. Here, we frame the task of providing explanations for the decisions of machine-learning models as a causal learning task, and train causal explanation (CXPlain) models that learn to estimate to what degree certain inputs cause outputs in another machine-learning model. CXPlain can, once trained, be used to explain the target model in little time, and enables the quantification of the uncertainty associated with its feature importance estimates via bootstrap ensembling. We present experiments that demonstrate that CXPlain is significantly more accurate and faster than existing model-agnostic methods for estimating feature importance. In addition, we confirm that the uncertainty estimates provided by CXPlain ensembles are strongly correlated with their ability to accurately estimate feature importance on held-out data.

To appear in Advances in Neural Information Processing Systems 2019

Cited by in corpus (27)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Explaining by Removing: A Unified Framework for Model Explanation
• Learning Counterfactual Representations for Estimating Individual Dose-Response Curves
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena
• CASTLE: Regularization via Auxiliary Causal Graph Discovery
• Self-Explaining Structures Improve NLP Models
• Reliable Post hoc Explanations: Modeling Uncertainty in Explainability
• Feature Removal Is a Unifying Principle for Model Explanation Methods
• Adversarial Infidelity Learning for Model Interpretation
• Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation
• Generative causal explanations of black-box classifiers
• CDT: Cascading Decision Trees for Explainable Reinforcement Learning
• Pair the Dots: Jointly Examining Training History and Test Stimuli for Model Interpretability
• Distal Explanations for Model-free Explainable Reinforcement Learning
• Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretations
• Contextual Prediction Difference Analysis for Explaining Individual Image Classifications
• Generative Interventions for Causal Learning
• Clinical Predictive Models for COVID-19: Systematic Study
• FastSHAP: Real-Time Shapley Value Estimation
• Shapley variable importance clouds for interpretable machine learning
• Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine Learning
• Explaining the Behavior of Black-Box Prediction Algorithms with Causal Learning
• Why Attentions May Not Be Interpretable?
• Visualizing Color-wise Saliency of Black-Box Image Classification Models
• Neural-to-Tree Policy Distillation with Policy Improvement Criterion
• Difference-in-Differences: Bridging Normalization and Disentanglement in PG-GAN
• Influence-Driven Explanations for Bayesian Network Classifiers