True to the Model or True to the Data?

Abstract

A variety of recent papers discuss the application of Shapley values, a concept for explaining coalitional games, for feature attribution in machine learning. However, the correct way to connect a machine learning model to a coalitional game has been a source of controversy. The two main approaches that have been proposed differ in the way that they condition on known features, using either (1) an interventional or (2) an observational conditional expectation. While previous work has argued that one of the two approaches is preferable in general, we argue that the choice is application dependent. Furthermore, we argue that the choice comes down to whether it is desirable to be true to the model or true to the data. We use linear models to investigate this choice. After deriving an efficient method for calculating observational conditional expectation Shapley values for linear models, we investigate how correlation in simulated data impacts the convergence of observational conditional expectation Shapley values. Finally, we present two real data examples that we consider to be representative of possible use cases for feature attribution -- (1) credit risk modeling and (2) biological discovery. We show how a different choice of value function performs better in each scenario, and how possible attributions are impacted by modeling choices.

References in corpus (3)

• Feature relevance quantification in explainable AI: A causal problem
• Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability
• Explaining black box decisions by Shapley cohort refinement

Cited by in corpus (18)

• Explaining a Series of Models by Propagating Shapley Values
• Explaining by Removing: A Unified Framework for Model Explanation
• Counterfactual Shapley Additive Explanations
• Fast TreeSHAP: Accelerating SHAP Value Computation for Trees
• Epistemic values in feature importance methods: Lessons from feminist epistemology
• Shapley Flow: A Graph-based Approach to Interpreting Model Predictions
• Fast Hierarchical Games for Image Explanations
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• Shapley explainability on the data manifold
• Wasserstein-based fairness interpretability framework for machine learning models
• Seven challenges for harmonizing explainability requirements
• The Shapley Value of coalition of variables provides better explanations
• groupShapley: Efficient prediction explanation with Shapley values for feature groups
• Data-driven advice for interpreting local and global model predictions in bioinformatics problems
• Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decomposition
• Marginal Contribution Feature Importance -- an Axiomatic Approach for The Natural Case
• Using Shapley Values and Variational Autoencoders to Explain Predictive Models with Dependent Mixed Features
• Model-agnostic bias mitigation methods with regressor distribution control for Wasserstein-based fairness metrics