Do Not Trust Additive Explanations
arXiv:1903.11420
Abstract
Explainable Artificial Intelligence (XAI)has received a great deal of attention recently. Explainability is being presented as a remedy for the distrust of complex and opaque models. Model agnostic methods such as LIME, SHAP, or Break Down promise instance-level interpretability for any complex machine learning model. But how faithful are these additive explanations? Can we rely on additive explanations for non-additive models? In this paper, we (1) examine the behavior of the most popular instance-level explanations under the presence of interactions, (2) introduce a new method that detects interactions for instance-level explanations, (3) perform a large scale benchmark to see how frequently additive explanations may be misleading.
arXiv comment15 pages
References in corpus (5)
- Learning Important Features Through Propagating Activation Differences
- OpenML: networked science in machine learning
- Consistent Individualized Feature Attribution for Tree Ensembles
- Adverse Events in Robotic Surgery: A Retrospective Study of 14 Years of FDA Data
- On the (In)fidelity and Sensitivity for Explanations
Cited by in corpus (7)
- Landscape of R packages for eXplainable Artificial Intelligence
- Transparency, Auditability and eXplainability of Machine Learning Models in Credit Scoring
- Explainable Software Defect Prediction: Are We There Yet?
- Understanding surrogate explanations: the interplay between complexity, fidelity and coverage
- What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversations
- Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanations
- Local Interpretability of Calibrated Prediction Models: A Case of Type 2 Diabetes Mellitus Screening Test