Visualizing and Understanding Convolutional Networks

Abstract

Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we address both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. We also perform an ablation study to discover the performance contribution from different model layers. This enables us to find model architectures that outperform Krizhevsky \etal on the ImageNet classification benchmark. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.

References in corpus (1)

• Improving neural networks by preventing co-adaptation of feature detectors

Cited by in corpus (183)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• Intriguing properties of neural networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• How transferable are features in deep neural networks?
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Learning Important Features Through Propagating Activation Differences
• Understanding Neural Networks Through Deep Visualization
• Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Brain's Ventral Visual Pathway
• Deeply-Supervised Nets
• Compressing Deep Convolutional Networks using Vector Quantization
• YouTube-8M: A Large-Scale Video Classification Benchmark
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• On the Number of Linear Regions of Deep Neural Networks
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• Convolutional Feature Masking for Joint Object and Stuff Segmentation
• The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
• Convolutional Neural Network-based Place Recognition
• Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
• Deep convolutional networks for pancreas segmentation in CT imaging
• A Comparative Study of Fruit Detection and Counting Methods for Yield Mapping in Apple Orchards
• CNN-based Segmentation of Medical Imaging Data
• Deep Learning for Tumor Classification in Imaging Mass Spectrometry
• DeepID-Net: multi-stage and deformable deep convolutional neural networks for object detection
• Deep Networks with Internal Selective Attention through Feedback Connections
• Deep learning for source camera identification on mobile devices
• Learning Contextual Dependencies with Convolutional Hierarchical Recurrent Neural Networks
• On the number of response regions of deep feed forward networks with piece-wise linear activations
• A learning-based method for solving ill-posed nonlinear inverse problems: a simulation study of Lung EIT
• Scale-Invariant Convolutional Neural Networks
• SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
• Deep Neural Network for Real-Time Autonomous Indoor Navigation
• Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping
• Modelling, Visualising and Summarising Documents with a Single Convolutional Neural Network
• Extraction of Salient Sentences from Labelled Documents
• DenseCap: Fully Convolutional Localization Networks for Dense Captioning
• Clinical Intervention Prediction and Understanding using Deep Networks
• Feature Representation in Convolutional Neural Networks
• DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
• On the Computational Efficiency of Training Neural Networks
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition
• One-Shot Adaptation of Supervised Deep Convolutional Models
• Toward quantitative fractography using convolutional neural networks
• Driving Scene Perception Network: Real-time Joint Detection, Depth Estimation and Semantic Segmentation
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Using Visual Analytics to Interpret Predictive Machine Learning Models
• End-to-End Parkinson Disease Diagnosis using Brain MR-Images by 3D-CNN
• Neuronal Synchrony in Complex-Valued Deep Networks
• Using Deep Learning to Localize Gravitational Wave Sources
• Transfer learning for radio galaxy classification
• Unsupervised feature learning by augmenting single images
• Have You Stolen My Model? Evasion Attacks Against Deep Neural Network Watermarking Techniques
• Fully Convolutional Networks for Chip-wise Defect Detection Employing Photoluminescence Images
• Deep learning approach for identification of HII regions during reionization in 21-cm observations
• A Convolutional Neural Network for Multiple Particle Identification in the MicroBooNE Liquid Argon Time Projection Chamber
• Stacked Quantizers for Compositional Vector Compression
• Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks
• On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models
• MoDeep: A Deep Learning Framework Using Motion Features for Human Pose Estimation
• MAMMO: A Deep Learning Solution for Facilitating Radiologist-Machine Collaboration in Breast Cancer Diagnosis
• On the Use of Neural Networks for Energy Reconstruction in High-granularity Calorimeters
• Score Function Features for Discriminative Learning: Matrix and Tensor Framework
• What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors
• Evaluation of convolutional neural networks using a large multi-subject P300 dataset
• Smoothed Geometry for Robust Attribution
• Deep Convolutional Networks are Hierarchical Kernel Machines
• Deep Co-Training for Semi-Supervised Image Recognition
• Matching-CNN Meets KNN: Quasi-Parametric Human Parsing
• Towards Frequency-Based Explanation for Robust CNN
• Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification
• Generic Deep Networks with Wavelet Scattering
• Fashioning with Networks: Neural Style Transfer to Design Clothes
• Challenging Environments for Traffic Sign Detection: Reliability Assessment under Inclement Conditions
• Flip-Rotate-Pooling Convolution and Split Dropout on Convolution Neural Networks for Image Classification
• Connecting optical morphology, environment, and HI mass fraction for low-redshift galaxies using deep learning
• Learning Temporal Pose Estimation from Sparsely-Labeled Videos
• Iris and periocular recognition in arabian race horses using deep convolutional neural networks
• ProtoAttend: Attention-Based Prototypical Learning
• Combining the Best of Graphical Models and ConvNets for Semantic Segmentation
• CondenseNeXt: An Ultra-Efficient Deep Neural Network for Embedded Systems
• A HMAX with LLC for visual recognition
• Fair Comparison: Quantifying Variance in Resultsfor Fine-grained Visual Categorization
• The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification
• Object Level Deep Feature Pooling for Compact Image Representation
• Deep Learning for identifying radiogenomic associations in breast cancer
• Neural network-based preprocessing to estimate the parameters of the X-ray emission of a single-temperature thermal plasma
• SelfieBoost: A Boosting Algorithm for Deep Learning
• Convolutional Models for Joint Object Categorization and Pose Estimation
• Generalizing multistain immunohistochemistry tissue segmentation using one-shot color deconvolution deep neural networks
• Multi-task Batch Reinforcement Learning with Metric Learning
• The PAU Survey: Background light estimation with deep learning techniques
• Real-world Object Recognition with Off-the-shelf Deep Conv Nets: How Many Objects can iCub Learn?
• Autoencoding sensory substitution
• Large Scale Business Discovery from Street Level Imagery
• Multi-scale Orderless Pooling of Deep Convolutional Activation Features
• Benanza: Automatic $\mu$Benchmark Generation to Compute "Lower-bound" Latency and Inform Optimizations of Deep Learning Models on GPUs
• Integrated perception with recurrent multi-task neural networks
• Improving Image Classification Robustness through Selective CNN-Filters Fine-Tuning
• Cyclic Boosting -- an explainable supervised machine learning algorithm
• XRAI: Better Attributions Through Regions
• Fast Neural Architecture Construction using EnvelopeNets
• Deep Epitomic Convolutional Neural Networks
• DeepPicker: a Deep Learning Approach for Fully Automated Particle Picking in Cryo-EM
• Detection of Premature Ventricular Contractions Using Densely Connected Deep Convolutional Neural Network with Spatial Pyramid Pooling Layer
• Detector Discovery in the Wild: Joint Multiple Instance and Representation Learning
• Out-of-Distribution Detection Using Neural Rendering Generative Models
• From Selective Deep Convolutional Features to Compact Binary Representations for Image Retrieval
• What are the visual features underlying human versus machine vision?
• Deep Learning for Metagenomic Data: using 2D Embeddings and Convolutional Neural Networks
• Complementary reinforcement learning towards explainable agents
• Weakly-Supervised Object Detection Learning through Human-Robot Interaction
• Do Vision Models Encode Object-Level Semantic Relatedness? A Cognitive Psychology-Inspired Benchmark
• Structured Feature Learning for Pose Estimation
• Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation
• Interpreting Interpretations: Organizing Attribution Methods by Criteria
• On monitoring development indicators using high resolution satellite images
• Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures
• Interpretable Neuron Structuring with Graph Spectral Regularization
• Human perception in computer vision
• 3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction
• Genetic Neural Architecture Search for automatic assessment of human sperm images
• Towards Interpretable Ensemble Learning for Image-based Malware Detection
• Compression of Deep Neural Networks for Image Instance Retrieval
• Spectral Analysis of Latent Representations
• CUNet: A Compact Unsupervised Network for Image Classification
• Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
• $N^4$-Fields: Neural Network Nearest Neighbor Fields for Image Transforms
• TSInsight: A local-global attribution framework for interpretability in time-series data
• Automatic Photo Adjustment Using Deep Neural Networks
• The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN
• Collaborative Receptive Field Learning
• GANMEX: One-vs-One Attributions Guided by GAN-based Counterfactual Explanation Baselines
• Convolutional Neural Network Models and Interpretability for the Anisotropic Reynolds Stress Tensor in Turbulent One-dimensional Flows
• MIASSR: An Approach for Medical Image Arbitrary Scale Super-Resolution
• Deep Net Triage: Analyzing the Importance of Network Layers via Structural Compression
• Gradually Updated Neural Networks for Large-Scale Image Recognition
• An Evolution of CNN Object Classifiers on Low-Resolution Images
• Counterfactual Generation with Knockoffs
• Analyzing Stability of Convolutional Neural Networks in the Frequency Domain
• Training on test data: Removing near duplicates in Fashion-MNIST
• Recurrent U-net: Deep learning to predict daily summertime ozone in the United States
• Enhancing Sound Texture in CNN-Based Acoustic Scene Classification
• Efficient On-the-fly Category Retrieval using ConvNets and GPUs
• Improving image generative models with human interactions
• Towards Deep Compositional Networks
• A Picture Tells a Thousand Words -- About You! User Interest Profiling from User Generated Visual Content
• Peek Inside the Closed World: Evaluating Autoencoder-Based Detection of DDoS to Cloud
• Neuron ranking -- an informed way to condense convolutional neural networks architecture
• Deconfusing intensity maps with neural networks
• ResIST: Layer-Wise Decomposition of ResNets for Distributed Training
• Global Context Networks
• Visual Explanation for Identification of the Brain Bases for Dyslexia on fMRI Data
• LAMVI-2: A Visual Tool for Comparing and Tuning Word Embedding Models
• Ten AI Stepping Stones for Cybersecurity
• Face Attribute Invertion
• Machine Vision in the Context of Robotics: A Systematic Literature Review
• Retinal Microvasculature as Biomarker for Diabetes and Cardiovascular Diseases
• Semantic Image Cropping
• A Generalization Theory based on Independent and Task-Identically Distributed Assumption
• Training Deep Neural Networks via Optimization Over Graphs
• Regularizing Explanations in Bayesian Convolutional Neural Networks
• DNNs as Layers of Cooperating Classifiers
• Similarities and differences between stimulus tuning in the inferotemporal visual cortex and convolutional networks
• Convergence Analysis of Gradient Descent Algorithms with Proportional Updates
• Automatically Segmenting the Left Atrium from Cardiac Images Using Successive 3D U-Nets and a Contour Loss
• Pixel-wise object tracking
• Subset Feature Learning for Fine-Grained Category Classification
• Saliency Supervision: An Intuitive and Effective Approach for Pain Intensity Regression
• Improving Attribution Methods by Learning Submodular Functions
• Per-Pixel Feedback for improving Semantic Segmentation
• Crowding in humans is unlike that in convolutional neural networks
• Collaborative creativity with Monte-Carlo Tree Search and Convolutional Neural Networks
• EmbNum: Semantic labeling for numerical values with deep metric learning
• Annotation Scaffolds for Object Modeling and Manipulation
• Mediated Experts for Deep Convolutional Networks
• A Distributed Deep Representation Learning Model for Big Image Data Classification
• Scientific Calculator for Designing Trojan Detectors in Neural Networks
• Analyzing Representations inside Convolutional Neural Networks
• Representaciones del aprendizaje reutilizando los gradientes de la retropropagacion