Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability

Abstract

Explaining AI systems is fundamental both to the development of high performing models and to the trust placed in them by their users. The Shapley framework for explainability has strength in its general applicability combined with its precise, rigorous foundation: it provides a common, model-agnostic language for AI explainability and uniquely satisfies a set of intuitive mathematical axioms. However, Shapley values are too restrictive in one significant regard: they ignore all causal structure in the data. We introduce a less restrictive framework, Asymmetric Shapley values (ASVs), which are rigorously founded on a set of axioms, applicable to any AI system, and flexible enough to incorporate any causal structure known to be respected by the data. We demonstrate that ASVs can (i) improve model explanations by incorporating causal information, (ii) provide an unambiguous test for unfair discrimination in model predictions, (iii) enable sequentially incremental explanations in time-series models, and (iv) support feature-selection studies without the need for model retraining.

arXiv commentTo appear in NeurIPS 2020; 9 pages, 2 figures, 2 appendices

References in corpus (12)

• Auto-Encoding Variational Bayes
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Learning Important Features Through Propagating Activation Differences
• Equality of Opportunity in Supervised Learning
• All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
• Counterfactual Fairness
• How to Explain Individual Classification Decisions
• Consistent Individualized Feature Attribution for Tree Ensembles
• Feature relevance quantification in explainable AI: A causal problem
• Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers
• Censoring Representations with an Adversary
• Shapley explainability on the data manifold

Cited by in corpus (8)

• Explainability in Music Recommender Systems
• True to the Model or True to the Data?
• Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models
• Shapley Flow: A Graph-based Approach to Interpreting Model Predictions
• Fast Hierarchical Games for Image Explanations
• Feature Removal Is a Unifying Principle for Model Explanation Methods
• Why model why? Assessing the strengths and limitations of LIME
• Improvement-Focused Causal Recourse (ICR)