Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks

Abstract

Recently, increasing attention has been drawn to the internal mechanisms of convolutional neural networks, and the reason why the network makes specific decisions. In this paper, we develop a novel post-hoc visual explanation method called Score-CAM based on class activation mapping. Unlike previous class activation mapping based approaches, Score-CAM gets rid of the dependence on gradients by obtaining the weight of each activation map through its forward passing score on target class, the final result is obtained by a linear combination of weights and activation maps. We demonstrate that Score-CAM achieves better visual performance and fairness for interpreting the decision making process. Our approach outperforms previous methods on both recognition and localization tasks, it also passes the sanity check. We also indicate its application as debugging tools. Official code has been released.

arXiv commentAccepted to CVPR 2020: Workshop on Fair, Data Efficient and Trusted Computer Vision

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• Learning Important Features Through Propagating Activation Differences
• VQA: Visual Question Answering
• SmoothGrad: removing noise by adding noise
• Sanity Checks for Saliency Maps
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models
• Tell Me Where to Look: Guided Attention Inference Network

Cited by in corpus (10)

• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• Audeo: Audio Generation for a Silent Performance Video
• LimitNet: Progressive, Content-Aware Image Offloading for Extremely Weak Devices & Networks
• From Labels to Priors in Capsule Endoscopy: A Prior Guided Approach for Improving Generalization with Few Labels
• XDeep: An Interpretation Tool for Deep Neural Networks
• RealSmileNet: A Deep End-To-End Network for Spontaneous and Posed Smile Recognition
• Weakly Supervised Minirhizotron Image Segmentation with MIL-CAM
• Cluster Activation Mapping with Applications to Medical Imaging
• How to Explain Neural Networks: an Approximation Perspective
• Efficient Modelling Across Time of Human Actions and Interactions