An unexpected unity among methods for interpreting model predictions

Abstract

Understanding why a model made a certain prediction is crucial in many data science fields. Interpretable predictions engender appropriate trust and provide insight into how the model may be improved. However, with large modern datasets the best accuracy is often achieved by complex models even experts struggle to interpret, which creates a tension between accuracy and interpretability. Recently, several methods have been proposed for interpreting predictions from complex models by estimating the importance of input features. Here, we present how a model-agnostic additive representation of the importance of input features unifies current methods. This representation is optimal, in the sense that it is the only set of additive values that satisfies important properties. We show how we can leverage these properties to create novel visual explanations of model predictions. The thread of unity that this representation weaves through the literature indicates that there are common principles to be learned about the interpretation of model predictions that apply in many scenarios.

Presented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems

References in corpus (2)

• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• "Why Should I Trust You?": Explaining the Predictions of Any Classifier

Cited by in corpus (22)

• Axiomatic Attribution for Deep Networks
• Learning Important Features Through Propagating Activation Differences
• Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
• Consistent feature attribution for tree ensembles
• Adversarial Examples on Graph Data: Deep Insights into Attack and Defense
• Explainable Artificial Intelligence Approaches: A Survey
• Parkinson's Disease Recognition Using SPECT Image and Interpretable AI: A Tutorial
• Stakeholders in Explainable AI
• MAGIX: Model Agnostic Globally Interpretable Explanations
• Global Aggregations of Local Explanations for Black Box models
• Contrastive Explanations with Local Foil Trees
• Why model why? Assessing the strengths and limitations of LIME
• Optimizing for Interpretability in Deep Neural Networks with Tree Regularization
• "Why Should I Trust Interactive Learners?" Explaining Interactive Queries of Classifiers to Users
• MeLIME: Meaningful Local Explanation for Machine Learning Models
• Fibres of Failure: Classifying errors in predictive processes
• Extracting Incentives from Black-Box Decisions
• Technical Report: Partial Dependence through Stratification
• Fuzzy Logic Interpretation of Quadratic Networks
• Information-theoretic Evolution of Model Agnostic Global Explanations
• Decorrelated Variable Importance
• Training Machine Learning Models by Regularizing their Explanations