Problems with Shapley-value-based explanations as feature importance measures

Abstract

Game-theoretic formulations of feature importance have become popular as a way to "explain" machine learning models. These methods define a cooperative game between the features of a model and distribute influence among these input elements using some form of the game's unique Shapley values. Justification for these methods rests on two pillars: their desirable mathematical properties, and their applicability to specific motivations for explanations. We show that mathematical problems arise when Shapley values are used for feature importance and that the solutions to mitigate these necessarily induce further complexity, such as the need for causal reasoning. We also draw on additional literature to argue that Shapley values do not provide explanations which suit human-centric goals of explainability.

arXiv commentAccepted to ICML 2020

References in corpus (3)

• Consistent Individualized Feature Attribution for Tree Ensembles
• Actionable Recourse in Linear Classification
• A Human-Grounded Evaluation of SHAP for Alert Processing

Cited by in corpus (21)

• Explaining a Series of Models by Propagating Shapley Values
• Impossibility Theorems for Feature Attribution
• Explainability in Music Recommender Systems
• Counterfactual Shapley Additive Explanations
• Shapley Counterfactual Credits for Multi-Agent Reinforcement Learning
• Fast TreeSHAP: Accelerating SHAP Value Computation for Trees
• Outcome-Explorer: A Causality Guided Interactive Visual Interface for Interpretable Algorithmic Decision Making
• Epistemic values in feature importance methods: Lessons from feminist epistemology
• Rational Shapley Values
• Designing Inherently Interpretable Machine Learning Models
• Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation
• On the Complexity of SHAP-Score-Based Explanations: Tractability via Knowledge Compilation and Non-Approximability Results
• Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
• Explaining classifiers to understand coarse-grained models
• Measuring the Driving Forces of Predictive Performance: Application to Credit Scoring
• On the overlooked issue of defining explanation objectives for local-surrogate explainers
• Extending LIME for Business Process Automation
• Seven challenges for harmonizing explainability requirements
• Towards Rigorous Interpretations: a Formalisation of Feature Attribution
• Evaluating the Values of Sources in Transfer Learning
• Do not explain without context: addressing the blind spot of model explanations