True to the Model or True to the Data?

Abstract

A variety of recent papers discuss the application of Shapley values, a concept for explaining coalitional games, for feature attribution in machine learning. However, the correct way to connect a machine learning model to a coalitional game has been a source of controversy. The two main approaches that have been proposed differ in the way that they condition on known features, using either (1) an interventional or (2) an observational conditional expectation. While previous work has argued that one of the two approaches is preferable in general, we argue that the choice is application dependent. Furthermore, we argue that the choice comes down to whether it is desirable to be true to the model or true to the data. We use linear models to investigate this choice. After deriving an efficient method for calculating observational conditional expectation Shapley values for linear models, we investigate how correlation in simulated data impacts the convergence of observational conditional expectation Shapley values. Finally, we present two real data examples that we consider to be representative of possible use cases for feature attribution -- (1) credit risk modeling and (2) biological discovery. We show how a different choice of value function performs better in each scenario, and how possible attributions are impacted by modeling choices.

References in corpus (1)

• Feature relevance quantification in explainable AI: A causal problem

Cited by in corpus (5)

• Epistemic values in feature importance methods: Lessons from feminist epistemology
• Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
• Seven challenges for harmonizing explainability requirements
• Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decomposition
• groupShapley: Efficient prediction explanation with Shapley values for feature groups