Visualizing Color-wise Saliency of Black-Box Image Classification Models
arXiv:2010.02468
Abstract
Image classification based on machine learning is being commonly used. However, a classification result given by an advanced method, including deep learning, is often hard to interpret. This problem of interpretability is one of the major obstacles in deploying a trained model in safety-critical systems. Several techniques have been proposed to address this problem; one of which is RISE, which explains a classification result by a heatmap, called a saliency map, which explains the significance of each pixel. We propose MC-RISE (Multi-Color RISE), which is an enhancement of RISE to take color information into account in an explanation. Our method not only shows the saliency of each pixel in a given image as the original RISE does, but the significance of color components of each pixel; a saliency map with color information is useful especially in the domain where the color information matters (e.g., traffic-sign recognition). We implemented MC-RISE and evaluate them using two datasets (GTSRB and ImageNet) to demonstrate the effectiveness of our methods in comparison with existing techniques for interpreting image classification results.
To appear in ACCV 2020
References in corpus (6)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- SmoothGrad: removing noise by adding noise
- Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch
- CXPlain: Causal Explanations for Model Interpretation under Uncertainty
- Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection