The Shapley Taylor Interaction Index

Abstract

The attribution problem, that is the problem of attributing a model's prediction to its base features, is well-studied. We extend the notion of attribution to also apply to feature interactions. The Shapley value is a commonly used method to attribute a model's prediction to its base features. We propose a generalization of the Shapley value called Shapley-Taylor index that attributes the model's prediction to interactions of subsets of features up to some size k. The method is analogous to how the truncated Taylor Series decomposes the function value at a certain point using its derivatives at a different point. In fact, we show that the Shapley Taylor index is equal to the Taylor Series of the multilinear extension of the set-theoretic behavior of the model. We axiomatize this method using the standard Shapley axioms -- linearity, dummy, symmetry and efficiency -- and an additional axiom that we call the interaction distribution axiom. This new axiom explicitly characterizes how interactions are distributed for a class of functions that model pure interaction. We contrast the Shapley-Taylor index against the previously proposed Shapley Interaction index (cf. [9]) from the cooperative game theory literature. We also apply the Shapley Taylor index to three models and identify interesting qualitative insights.

References in corpus (7)

• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• Learning Important Features Through Propagating Activation Differences
• How to Explain Individual Classification Decisions
• Consistent Individualized Feature Attribution for Tree Ensembles
• Hierarchical interpretations for neural network predictions
• Influence in Classification via Cooperative Game Theory

Cited by in corpus (6)

• Explaining Explanations: Axiomatic Feature Interactions for Deep Networks
• How does this interaction affect me? Interpretable attribution for feature interactions
• PredDiff: Explanations and Interactions from Conditional Expectations
• Technical Note: Game-Theoretic Interactions of Different Orders
• High Dimensional Model Explanations: an Axiomatic Approach
• Interpretable Artificial Intelligence through the Lens of Feature Interaction