Grad-CAM: Why did you say that?

Abstract

We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradient-weighted Class Activation Mapping (Grad-CAM), uses class-specific gradient information to localize important regions. These localizations are combined with existing pixel-space visualizations to create a novel high-resolution and class-discriminative visualization called Guided Grad-CAM. These methods help better understand CNN-based models, including image captioning and visual question answering (VQA) models. We evaluate our visual explanations by measuring their ability to discriminate between classes, to inspire trust in humans, and their correlation with occlusion maps. Grad-CAM provides a new way to understand CNN-based models. We have released code, an online demo hosted on CloudCV, and a full version of this extended abstract.

arXiv commentPresented at NIPS 2016 Workshop on Interpretable Machine Learning in Complex Systems. This is an extended abstract version of arXiv:1610.02391 (CVPR format)

References in corpus (2)

• Striving for Simplicity: The All Convolutional Net
• Hierarchical Question-Image Co-Attention for Visual Question Answering

Cited by in corpus (49)

• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases
• Experimental velocity data estimation for imperfect particle images using machine learning
• A Deep Multi-Level Attentive network for Multimodal Sentiment Analysis
• Generalization techniques of neural networks for fluid flow estimation
• Explainable Artificial Intelligence for Process Mining: A General Overview and Application of a Novel Local Explanation Approach for Predictive Process Monitoring
• Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms
• On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors
• Deep Weighted Averaging Classifiers
• A deep convolutional neural network for classification of Aedes albopictus mosquitoes
• MAGIX: Model Agnostic Globally Interpretable Explanations
• Dropout Feature Ranking for Deep Learning Models
• Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos
• Neural Network Attributions: A Causal Perspective
• Robust Tumor Localization with Pyramid Grad-CAM
• Towards falsifiable interpretability research
• Meta-trained agents implement Bayes-optimal agents
• Interpreting Deep Classifier by Visual Distillation of Dark Knowledge
• An Introduction to Deep Visual Explanation
• Deep machine learning for meteor monitoring: advances with transfer learning and gradient-weighted class activation mapping
• Bias-based Universal Adversarial Patch Attack for Automatic Check-out
• Capsule Networks for Protein Structure Classification and Prediction
• Optimizing for Interpretability in Deep Neural Networks with Tree Regularization
• Regional Tree Regularization for Interpretability in Black Box Models
• Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations
• Explainable Semantic Federated Learning Enabled Industrial Edge Network for Fire Surveillance
• Learning Fair Rule Lists
• Deep Attentional Structured Representation Learning for Visual Recognition
• Web Applicable Computer-aided Diagnosis of Glaucoma Using Deep Learning
• Fibres of Failure: Classifying errors in predictive processes
• Fast and Reliable Probabilistic Face Embeddings in the Wild
• A Simple Framework for Uncertainty in Contrastive Learning
• Concept Learners for Few-Shot Learning
• Genetic Neural Architecture Search for automatic assessment of human sperm images
• Detecting Adversarial Perturbations with Saliency
• Using KL-divergence to focus Deep Visual Explanation
• Zap: Making Predictions Based on Online User Behavior
• Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine Learning
• Optimal Piecewise Local-Linear Approximations
• Scaling Symbolic Methods using Gradients for Neural Model Explanation
• SCALP -- Supervised Contrastive Learning for Cardiopulmonary Disease Classification and Localization in Chest X-rays using Patient Metadata
• Information-theoretic Evolution of Model Agnostic Global Explanations
• Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach
• Training Machine Learning Models by Regularizing their Explanations
• Improving Attribution Methods by Learning Submodular Functions
• Survival-oriented embeddings for improving accessibility to complex data structures
• Deep Learning for Wireless Communications
• e-QRAQ: A Multi-turn Reasoning Dataset and Simulator with Explanations
• Controversy Rules - Discovering Regions Where Classifiers (Dis-)Agree Exceptionally