NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
arXiv:1909.04200
Abstract
The problem of explaining deep learning models, and model predictions generally, has attracted intensive interest recently. Many successful approaches forgo global approximations in order to provide more faithful local interpretations of the model's behavior. LIME develops multiple interpretable models, each approximating a large neural network on a small region of the data manifold and SP-LIME aggregates the local models to form a global interpretation. Extending this line of research, we propose a simple yet effective method, NormLIME for aggregating local models into global and class-specific interpretations. A human user study strongly favored class-specific interpretations created by NormLIME to other feature importance metrics. Numerical experiments confirm that NormLIME is effective at recognizing important features.
References in corpus (8)
- Axiomatic Attribution for Deep Networks
- Learning Important Features Through Propagating Activation Differences
- Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
- Understanding Black-box Predictions via Influence Functions
- SmoothGrad: removing noise by adding noise
- How to Explain Individual Classification Decisions
- Sanity Checks for Saliency Maps
- Revisiting the Importance of Individual Units in CNNs via Ablation
Cited by in corpus (7)
- HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks
- LIMEcraft: Handcrafted superpixel selection and inspection for Visual eXplanations
- MeLIME: Meaningful Local Explanation for Machine Learning Models
- Cosmic topology. Part IVa. Classification of manifolds using machine learning: a case study with small toroidal universes
- SurvLIME-Inf: A simplified modification of SurvLIME for explanation of machine learning survival models
- An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
- Towards Understanding the Effectiveness of Attention Mechanism