Explaining a Series of Models by Propagating Shapley Values

Abstract

Local feature attribution methods are increasingly used to explain complex machine learning models. However, current methods are limited because they are extremely expensive to compute or are not capable of explaining a distributed series of models where each model is owned by a separate institution. The latter is particularly important because it often arises in finance where explanations are mandated. Here, we present DeepSHAP, a tractable method to propagate local feature attributions through complex series of models based on a connection to the Shapley value. We evaluate DeepSHAP across biological, health, and financial datasets to show that it provides equally salient explanations an order of magnitude faster than existing model-agnostic attribution techniques and demonstrate its use in an important distributed series of models setting.

References in corpus (2)

• Problems with Shapley-value-based explanations as feature importance measures
• True to the Model or True to the Data?

Cited by in corpus (7)

• CircleZ: Reliable Photometric redshifts for AGN computed using only photometry from Legacy Survey Imaging for DESI
• Development and Validation of a Deep-Learning Model for Differential Treatment Benefit Prediction for Adults with Major Depressive Disorder Deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Study
• Sparse learned kernels for interpretable and efficient medical time series processing
• A Comprehensive Study of Shapley Value in Data Analytics
• On the interpretability of neural network decoders
• UbiQVision: Quantifying Uncertainty in XAI for Image Recognition
• Prospector Heads: Generalized Feature Attribution for Large Models & Data