Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images

Abstract

Image representations, from SIFT and bag of visual words to Convolutional Neural Networks (CNNs) are a crucial component of almost all computer vision systems. However, our understanding of them remains limited. In this paper we study several landmark representations, both shallow and deep, by a number of complementary visualization techniques. These visualizations are based on the concept of "natural pre-image", namely a natural-looking image whose representation has some notable property. We study in particular three such visualizations: inversion, in which the aim is to reconstruct an image from its representation, activation maximization, in which we search for patterns that maximally stimulate a representation component, and caricaturization, in which the visual patterns that a representation detects in an image are exaggerated. We pose these as a regularized energy-minimization framework and demonstrate its generality and effectiveness. In particular, we show that this method can invert representations such as HOG more accurately than recent alternatives while being applicable to CNNs too. Among our findings, we show that several layers in CNNs retain photographically accurate information about the image, with different degrees of geometric and photometric invariance.

arXiv commentA substantially extended version of http://www.robots.ox.ac.uk/~vedaldi/assets/pubs/mahendran15understanding.pdf. arXiv admin note: text overlap with arXiv:1412.0035

References in corpus (7)

• Intriguing properties of neural networks
• ADADELTA: An Adaptive Learning Rate Method
• Understanding Neural Networks Through Deep Visualization
• OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Texture Synthesis Using Convolutional Neural Networks
• A bi-level view of inpainting - based image compression

Cited by in corpus (101)

• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
• Interpretable Explanations of Black Boxes by Meaningful Perturbation
• Deep Image Prior
• Do Convolutional Neural Networks Learn Class Hierarchy?
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
• Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models
• Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs
• Neural Style Transfer: A Review
• Visualizing Convolutional Neural Network Protein-Ligand Scoring
• Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
• Explainable Diabetic Retinopathy Detection and Retinal Image Generation
• Bilinear CNNs for Fine-grained Visual Recognition
• i-RevNet: Deep Invertible Networks
• Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation
• How convolutional neural network see the world - A survey of convolutional neural network visualization methods
• Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
• Stacked Generative Adversarial Networks
• Survey of XAI in digital pathology
• Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos
• Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval
• An Analysis of Pre-Training on Object Detection
• Explainable Machine Learning with Prior Knowledge: An Overview
• See through Gradients: Image Batch Recovery via GradInversion
• Opening the black box of neural nets: case studies in stop/top discrimination
• Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment
• Deep Motif Dashboard: Visualizing and Understanding Genomic Sequences Using Deep Neural Networks
• Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation
• Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study
• Interpreting Super-Resolution Networks with Local Attribution Maps
• Metamorphic Testing for Object Detection Systems
• Evolutionary approaches to explainable machine learning
• Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
• Synthetic to Real Adaptation with Generative Correlation Alignment Networks
• A Novel Convolutional Neural Network for Image Steganalysis with Shared Normalization
• Deep multi-stations weather forecasting: explainable recurrent convolutional neural networks
• Generalizing multistain immunohistochemistry tissue segmentation using one-shot color deconvolution deep neural networks
• Sparse Oblique Decision Trees: A Tool to Understand and Manipulate Neural Net Features
• Understanding Individual Decisions of CNNs via Contrastive Backpropagation
• Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents
• Understanding the Impact of Semi-numeric Reionization Models when using CNNs
• Towards Interpretable Face Recognition
• Examining the Benefits of Capsule Neural Networks
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• Learning from Higher-Layer Feature Visualizations
• Rethinking the Role of Gradient-Based Attribution Methods for Model Interpretability
• Neural Network Memorization Dissection
• Towards Lifelong Self-Supervision: A Deep Learning Direction for Robotics
• Visualizing and Understanding Deep Neural Networks in CTR Prediction
• Egocentric Basketball Motion Planning from a Single First-Person Image
• Face Attributes as Cues for Deep Face Recognition Understanding
• A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
• Learning Interpretable Concept Groups in CNNs
• "I know it when I see it". Visualization and Intuitive Interpretability
• Detection of exomoons in simulated light curves with a regularized convolutional neural network
• On monitoring development indicators using high resolution satellite images
• Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
• Autonomous Vehicles Meet the Physical World: RSS, Variability, Uncertainty, and Proving Safety (Expanded Version)
• Path Analysis for Effective Fault Localization in Deep Neural Networks
• Geometry of Deep Convolutional Networks
• Tuning Convolutional Spiking Neural Network with Biologically-plausible Reward Propagation
• Decision Explanation and Feature Importance for Invertible Networks
• On Learning Semantic Representations for Million-Scale Free-Hand Sketches
• Model Interpretability and Rationale Extraction by Input Mask Optimization
• Leveraging Activation Maximization and Generative Adversarial Training to Recognize and Explain Patterns in Natural Areas in Satellite Imagery
• Designing Counterfactual Generators using Deep Model Inversion
• TopoAct: Visually Exploring the Shape of Activations in Deep Learning
• Towards Learning a Self-inverse Network for Bidirectional Image-to-image Translation
• Self-Regression Learning for Blind Hyperspectral Image Fusion Without Label
• Diverse feature visualizations reveal invariances in early layers of deep neural networks
• Vulnerability of Face Recognition Systems Against Composite Face Reconstruction Attack
• Sampling for Deep Learning Model Diagnosis (Technical Report)
• RNN-based Generative Model for Fine-Grained Sketching
• Deep Epitome for Unravelling Generalized Hamming Network: A Fuzzy Logic Interpretation of Deep Learning
• Building a Telescope to Look Into High-Dimensional Image Spaces
• TRIGS: Trojan Identification from Gradient-based Signatures
• Every Filter Extracts A Specific Texture In Convolutional Neural Networks
• A Categorisation of Post-hoc Explanations for Predictive Models
• Dirichlet Pruning for Neural Network Compression
• Separation of time scales and direct computation of weights in deep neural networks
• Visualizing Deep Learning-based Radio Modulation Classifier
• Inverting and Understanding Object Detectors
• Understanding Regularization to Visualize Convolutional Neural Networks
• Collaborative creativity with Monte-Carlo Tree Search and Convolutional Neural Networks
• Cosmic velocity, density and halo mass function: Insights from deep learning
• MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing
• Visualizing and Describing Fine-grained Categories as Textures
• Interpreting Deep Neural Networks with Relative Sectional Propagation by Analyzing Comparative Gradients and Hostile Activations
• Weakly Supervised Recovery of Semantic Attributes
• Diagnostic Visualization for Deep Neural Networks Using Stochastic Gradient Langevin Dynamics
• Understanding and Improving Kernel Local Descriptors
• Data-driven Regularization via Racecar Training for Generalizing Neural Networks
• An Overview of Computational Approaches for Interpretation Analysis
• Constraint-free Natural Image Reconstruction from fMRI Signals Based on Convolutional Neural Network
• On Equivariant and Invariant Learning of Object Landmark Representations
• Hierarchical Neural Representation of Dreamed Objects Revealed by Brain Decoding with Deep Neural Network Features
• LOTS about Attacking Deep Features
• Learning Realistic Patterns from Unrealistic Stimuli: Generalization and Data Anonymization
• A Framework to Learn with Interpretation
• Interpretable BoW Networks for Adversarial Example Detection