Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Abstract

This paper addresses the visualisation of image classification models, learnt using deep Convolutional Networks (ConvNets). We consider two visualisation techniques, based on computing the gradient of the class score with respect to the input image. The first one generates an image, which maximises the class score [Erhan et al., 2009], thus visualising the notion of the class, captured by a ConvNet. The second technique computes a class saliency map, specific to a given image and class. We show that such maps can be employed for weakly supervised object segmentation using classification ConvNets. Finally, we establish the connection between the gradient-based ConvNet visualisation methods and deconvolutional networks [Zeiler et al., 2013].

References in corpus (1)

• Visualizing and Understanding Convolutional Networks

Cited by in corpus (78)

• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Learning Important Features Through Propagating Activation Differences
• Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning
• Dueling Network Architectures for Deep Reinforcement Learning
• Understanding Neural Networks Through Deep Visualization
• Understanding Black-box Predictions via Influence Functions
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• A trans-disciplinary review of deep learning research for water resources scientists
• Explaining Explanations in AI
• Towards Explainable Artificial Intelligence
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Multi-task Self-Supervised Learning for Human Activity Detection
• Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability
• Rice Diseases Detection and Classification Using Attention Based Neural Network and Bayesian Optimization
• Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification
• Distilling a Neural Network Into a Soft Decision Tree
• Crop Yield Prediction Integrating Genotype and Weather Variables Using Deep Learning
• Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification
• Data-driven medium-range weather prediction with a Resnet pretrained on climate simulations: A new model for WeatherBench
• Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping
• Detecting total hip replacement prosthesis design on preoperative radiographs using deep convolutional neural network
• TimeSHAP: Explaining Recurrent Models through Sequence Perturbations
• Investigating the influence of noise and distractors on the interpretation of neural networks
• Interpretable deep learning for guided structure-property explorations in photovoltaics
• Explainability in Music Recommender Systems
• The relationship between fine galaxy stellar morphology and star formation activity in cosmological simulations: a deep learning view
• Temporal Graph Convolutional Networks for Automatic Seizure Detection
• On Tackling Explanation Redundancy in Decision Trees
• Compute and memory efficient universal sound source separation
• Visualizing Deep Networks by Optimizing with Integrated Gradients
• A Radiomics-Boosted Deep-Learning Model for COVID-19 and Non-COVID-19 Pneumonia Classification Using Chest X-ray Image
• Defining Locality for Surrogates in Post-hoc Interpretablity
• Deep Learning with Functional Inputs
• Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud
• Interpretability of a Deep Learning Model in the Application of Cardiac MRI Segmentation with an ACDC Challenge Dataset
• Extracting the cold neutral medium from HI emission with deep learning: Implications for Galactic foregrounds at high latitude
• The Role of Machine Learning in the Next Decade of Cosmology
• A Deep Learning Approach for Pose Estimation from Volumetric OCT Data
• Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks
• White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks
• When and How to Fool Explainable Models (and Humans) with Adversarial Examples
• Amortized Variational Inference: A Systematic Review
• DeepMiner: Discovering Interpretable Representations for Mammogram Classification and Explanation
• Explaining deep learning of galaxy morphology with saliency mapping
• Investigating the significance of adversarial attacks and their relation to interpretability for radar-based human activity recognition systems
• CELNet: Evidence Localization for Pathology Images using Weakly Supervised Learning
• DeepLofargram: A Deep Learning based Fluctuating Dim Frequency Line Detection and Recovery
• Connecting optical morphology, environment, and HI mass fraction for low-redshift galaxies using deep learning
• LCA: Loss Change Allocation for Neural Network Training
• Performance of GAN-based augmentation for deep learning COVID-19 image classification
• Deeply Explain CNN via Hierarchical Decomposition
• A Gradient Mapping Guided Explainable Deep Neural Network for Extracapsular Extension Identification in 3D Head and Neck Cancer Computed Tomography Images
• ContextMix: A context-aware data augmentation method for industrial visual inspection systems
• Visualizing and Understanding Sum-Product Networks
• Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models
• Why do These Match? Explaining the Behavior of Image Similarity Models
• Picasso: A Modular Framework for Visualizing the Learning Process of Neural Network Image Classifiers
• Explaining Neural Networks Semantically and Quantitatively
• Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
• Adaptive wavelet distillation from neural networks through interpretations
• Exploring complex pattern formation with convolutional neural networks
• FovEx: Human-Inspired Explanations for Vision Transformers and Convolutional Neural Networks
• Generating Post-hoc Explanations for Skip-gram-based Node Embeddings by Identifying Important Nodes with Bridgeness
• Challenges for cognitive decoding using deep learning methods
• Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces
• The Impact of Feature Representation on the Accuracy of Photonic Neural Networks
• Finding and Fixing Spurious Patterns with Explanations
• Interpreting Face Inference Models using Hierarchical Network Dissection
• Explaining AlphaGo: Interpreting Contextual Effects in Neural Networks
• Benchmark Visual Question Answer Models by using Focus Map
• Regularizing Black-box Models for Improved Interpretability (HILL 2019 Version)
• The Computational Complexity of Understanding Network Decisions
• Example-Based Concept Analysis Framework for Deep Weather Forecast Models
• Inferring halo masses with Graph Neural Networks
• Astro2020 Science White Paper: The Next Decade of Astroinformatics and Astrostatistics
• Inferring Thunderstorm Occurrence from Vertical Profiles of Convection-Permitting Simulations: Physical Insights from a Physical Deep Learning Model
• Fine-Grained Categorization via CNN-Based Automatic Extraction and Integration of Object-Level and Part-Level Features
• Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition