Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance

Abstract

At the core of interpretable machine learning is the question of whether humans are able to make accurate predictions about a model's behavior. Assumed in this question are three properties of the interpretable output: coverage, precision, and effort. Coverage refers to how often humans think they can predict the model's behavior, precision to how accurate humans are in those predictions, and effort is either the up-front effort required in interpreting the model, or the effort required to make predictions about a model's behavior. In this work, we propose anchor-LIME (aLIME), a model-agnostic technique that produces high-precision rule-based explanations for which the coverage boundaries are very clear. We compare aLIME to linear LIME with simulated experiments, and demonstrate the flexibility of aLIME with qualitative examples from a variety of domains and tasks.

Presented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems

References in corpus (1)

• Model-Agnostic Interpretability of Machine Learning

Cited by in corpus (8)

• Explanations of model predictions with live and breakDown packages
• To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods
• Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods
• Interpretable machine learning applied to on-farm biosecurity and porcine reproductive and respiratory syndrome virus
• Why model why? Assessing the strengths and limitations of LIME
• Understanding Individual Decisions of CNNs via Contrastive Backpropagation
• On Tractable Representations of Binary Neural Networks
• Fibres of Failure: Classifying errors in predictive processes