Why model why? Assessing the strengths and limitations of LIME
arXiv:2012.00093
Abstract
When it comes to complex machine learning models, commonly referred to as black boxes, understanding the underlying decision making process is crucial for domains such as healthcare and financial services, and also when it is used in connection with safety critical systems such as autonomous vehicles. As such interest in explainable artificial intelligence (xAI) tools and techniques has increased in recent years. However, the effectiveness of existing xAI frameworks, especially concerning algorithms that work with data as opposed to images, is still an open research question. In order to address this gap, in this paper we examine the effectiveness of the Local Interpretable Model-Agnostic Explanations (LIME) xAI framework, one of the most popular model agnostic frameworks found in the literature, with a specific focus on its performance in terms of making tabular models more interpretable. In particular, we apply several state of the art machine learning algorithms on a tabular dataset, and demonstrate how LIME can be used to supplement conventional performance assessment methods. In addition, we evaluate the understandability of the output produced by LIME both via a usability study, involving participants who are not familiar with LIME, and its overall usability via an assessment framework, which is derived from the International Organisation for Standardisation 9241-11:1998 standard.
arXiv comment13 pages, 4 figures
References in corpus (19)
- Axiomatic Attribution for Deep Networks
- How to Explain Individual Classification Decisions
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
- "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach
- Local Rule-Based Explanations of Black Box Decision Systems
- Model Agnostic Supervised Local Explanations
- An unexpected unity among methods for interpreting model predictions
- Interpretability via Model Extraction
- Model-Agnostic Counterfactual Explanations for Consequential Decisions
- Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance
- Machine Learning Model Interpretability for Precision Medicine
- TreeView: Peeking into Deep Neural Networks Via Feature-Space Partitioning
- Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)
- MAGIX: Model Agnostic Globally Interpretable Explanations
- L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data
- Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability
- Local Interpretable Model-agnostic Explanations of Bayesian Predictive Models via Kullback-Leibler Projections
- Producing radiologist-quality reports for interpretable artificial intelligence
- TED: Teaching AI to Explain its Decisions