Quantitative Evaluations on Saliency Methods: An Experimental Study
arXiv:2012.15616
Abstract
It has been long debated that eXplainable AI (XAI) is an important topic, but it lacks rigorous definition and fair metrics. In this paper, we briefly summarize the status quo of the metrics, along with an exhaustive experimental study based on them, including faithfulness, localization, false-positives, sensitivity check, and stability. With the experimental results, we conclude that among all the methods we compare, no single explanation method dominates others in all metrics. Nonetheless, Gradient-weighted Class Activation Mapping (Grad-CAM) and Randomly Input Sampling for Explanation (RISE) perform fairly well in most of the metrics. Utilizing a set of filtered metrics, we further present a case study to diagnose the classification bases for models. While providing a comprehensive experimental study of metrics, we also examine measuring factors that are missed in current metrics and hope this valuable work could serve as a guide for future research.
arXiv comment14 pages, 16 figures
References in corpus (13)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- A Unified Approach to Interpreting Model Predictions
- Towards A Rigorous Science of Interpretable Machine Learning
- Striving for Simplicity: The All Convolutional Net
- Sanity Checks for Saliency Maps
- Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
- "Why Should I Trust You?": Explaining the Predictions of Any Classifier
- Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
- Metrics for Explainable AI: Challenges and Prospects
- Explanations can be manipulated and geometry is to blame
- Benchmarking Attribution Methods with Relative Feature Importance
- On the (In)fidelity and Sensitivity for Explanations
- Evaluating Explanation Without Ground Truth in Interpretable Machine Learning