Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images

Abstract

Image representations, from SIFT and bag of visual words to Convolutional Neural Networks (CNNs) are a crucial component of almost all computer vision systems. However, our understanding of them remains limited. In this paper we study several landmark representations, both shallow and deep, by a number of complementary visualization techniques. These visualizations are based on the concept of "natural pre-image", namely a natural-looking image whose representation has some notable property. We study in particular three such visualizations: inversion, in which the aim is to reconstruct an image from its representation, activation maximization, in which we search for patterns that maximally stimulate a representation component, and caricaturization, in which the visual patterns that a representation detects in an image are exaggerated. We pose these as a regularized energy-minimization framework and demonstrate its generality and effectiveness. In particular, we show that this method can invert representations such as HOG more accurately than recent alternatives while being applicable to CNNs too. Among our findings, we show that several layers in CNNs retain photographically accurate information about the image, with different degrees of geometric and photometric invariance.

A substantially extended version of http://www.robots.ox.ac.uk/~vedaldi/assets/pubs/mahendran15understanding.pdf. arXiv admin note: text overlap with arXiv:1412.0035

References in corpus (2)

• ADADELTA: An Adaptive Learning Rate Method
• Understanding Neural Networks Through Deep Visualization

Cited by in corpus (100)

• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Interpretable Explanations of Black Boxes by Meaningful Perturbation
• Deep Image Prior
• Do Convolutional Neural Networks Learn Class Hierarchy?
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
• Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models
• Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs
• Neural Style Transfer: A Review
• Visualizing Convolutional Neural Network Protein-Ligand Scoring
• Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
• Explainable Diabetic Retinopathy Detection and Retinal Image Generation
• i-RevNet: Deep Invertible Networks
• Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation
• How convolutional neural network see the world - A survey of convolutional neural network visualization methods
• Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
• Stacked Generative Adversarial Networks
• Survey of XAI in digital pathology
• Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos
• Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval
• An Analysis of Pre-Training on Object Detection
• Explainable Machine Learning with Prior Knowledge: An Overview
• See through Gradients: Image Batch Recovery via GradInversion
• Opening the black box of neural nets: case studies in stop/top discrimination
• Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment
• Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks
• Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation
• Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study
• Metamorphic Testing for Object Detection Systems
• Interpreting Super-Resolution Networks with Local Attribution Maps
• Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
• Evolutionary approaches to explainable machine learning
• A Novel Convolutional Neural Network for Image Steganalysis with Shared Normalization
• Synthetic to Real Adaptation with Generative Correlation Alignment Networks
• Deep multi-stations weather forecasting: explainable recurrent convolutional neural networks
• Sparse Oblique Decision Trees: A Tool to Understand and Manipulate Neural Net Features
• Generalizing multistain immunohistochemistry tissue segmentation using one-shot color deconvolution deep neural networks
• Understanding Individual Decisions of CNNs via Contrastive Backpropagation
• Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents
• Understanding the Impact of Semi-numeric Reionization Models when using CNNs
• Examining the Benefits of Capsule Neural Networks
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• Learning from Higher-Layer Feature Visualizations
• Rethinking the Role of Gradient-Based Attribution Methods for Model Interpretability
• Towards Interpretable Face Recognition
• Neural Network Memorization Dissection
• Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics
• Visualizing and Understanding Deep Neural Networks in CTR Prediction
• Egocentric Basketball Motion Planning from a Single First-Person Image
• On monitoring development indicators using high resolution satellite images
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• Face Attributes as Cues for Deep Face Recognition Understanding
• Detection of exomoons in simulated light curves with a regularized convolutional neural network
• Learning Interpretable Concept Groups in CNNs
• "I know it when I see it". Visualization and Intuitive Interpretability
• Autonomous Vehicles Meet the Physical World: RSS, Variability, Uncertainty, and Proving Safety (Expanded Version)
• Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
• Path Analysis for Effective Fault Localization in Deep Neural Networks
• On Learning Semantic Representations for Million-Scale Free-Hand Sketches
• Geometry of Deep Convolutional Networks
• Tuning Convolutional Spiking Neural Network with Biologically-plausible Reward Propagation
• Decision Explanation and Feature Importance for Invertible Networks
• TopoAct: Visually Exploring the Shape of Activations in Deep Learning
• Deep Epitome for Unravelling Generalized Hamming Network: A Fuzzy Logic Interpretation of Deep Learning
• Towards Learning a Self-inverse Network for Bidirectional Image-to-image Translation
• Model Interpretability and Rationale Extraction by Input Mask Optimization
• Leveraging Activation Maximization and Generative Adversarial Training to Recognize and Explain Patterns in Natural Areas in Satellite Imagery
• Diverse feature visualizations reveal invariances in early layers of deep neural networks
• Designing Counterfactual Generators using Deep Model Inversion
• Vulnerability of Face Recognition Systems Against Composite Face Reconstruction Attack
• Sampling for Deep Learning Model Diagnosis (Technical Report)
• Self-Regression Learning for Blind Hyperspectral Image Fusion Without Label
• RNN-based Generative Model for Fine-Grained Sketching
• Building a Telescope to Look Into High-Dimensional Image Spaces
• Every Filter Extracts A Specific Texture In Convolutional Neural Networks
• A Categorisation of Post-hoc Explanations for Predictive Models
• Dirichlet Pruning for Neural Network Compression
• Inverting and Understanding Object Detectors
• TRIGS: Trojan Identification from Gradient-based Signatures
• Visualizing Deep Learning-based Radio Modulation Classifier
• Understanding Regularization to Visualize Convolutional Neural Networks
• Separation of time scales and direct computation of weights in deep neural networks
• Diagnostic Visualization for Deep Neural Networks Using Stochastic Gradient Langevin Dynamics
• Hierarchical Neural Representation of Dreamed Objects Revealed by Brain Decoding with Deep Neural Network Features
• On Equivariant and Invariant Learning of Object Landmark Representations
• Learning Realistic Patterns from Unrealistic Stimuli: Generalization and Data Anonymization
• A Framework to Learn with Interpretation
• MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing
• Weakly Supervised Recovery of Semantic Attributes
• Cosmic velocity, density and halo mass function: Insights from deep learning
• Interpreting Deep Neural Networks with Relative Sectional Propagation by Analyzing Comparative Gradients and Hostile Activations
• Data-driven Regularization via Racecar Training for Generalizing Neural Networks
• Interpretable BoW Networks for Adversarial Example Detection
• Visualizing and Describing Fine-grained Categories as Textures
• Collaborative creativity with Monte-Carlo Tree Search and Convolutional Neural Networks
• Constraint-free Natural Image Reconstruction from fMRI Signals Based on Convolutional Neural Network
• LOTS about Attacking Deep Features
• An Overview of Computational Approaches for Interpretation Analysis
• Understanding and Improving Kernel Local Descriptors