Revisiting the Importance of Individual Units in CNNs via Ablation

Abstract

We revisit the importance of the individual units in Convolutional Neural Networks (CNNs) for visual recognition. By conducting unit ablation experiments on CNNs trained on large scale image datasets, we demonstrate that, though ablating any individual unit does not hurt overall classification accuracy, it does lead to significant damage on the accuracy of specific classes. This result shows that an individual unit is specialized to encode information relevant to a subset of classes. We compute the correlation between the accuracy drop under unit ablation and various attributes of an individual unit such as class selectivity and weight L1 norm. We confirm that unit attributes such as class selectivity are a poor predictor for impact on overall accuracy as found previously in recent work \cite{morcos2018importance}. However, our results show that class selectivity along with other attributes are good predictors of the importance of one unit to individual classes. We evaluate the impact of random rotation, batch normalization, and dropout to the importance of units to specific classes. Our results show that units with high selectivity play an important role in network classification power at the individual class level. Understanding and interpreting the behavior of these units is necessary and meaningful.

References in corpus (7)

• Intriguing properties of neural networks
• Pruning Filters for Efficient ConvNets
• CNN Features off-the-shelf: an Astounding Baseline for Recognition
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• On the importance of single directions for generalization
• DSD: Dense-Sparse-Dense Training for Deep Neural Networks
• Convergent Learning: Do different neural networks learn the same representations?

Cited by in corpus (29)

• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• Understanding the Role of Individual Units in a Deep Neural Network
• Understanding trained CNNs by indexing neuron selectivity
• NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
• Identifying and Controlling Important Neurons in Neural Machine Translation
• Towards falsifiable interpretability research
• Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation
• Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena
• Distilling Critical Paths in Convolutional Neural Networks
• Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs
• Few-shot Object Detection via Feature Reweighting
• Functionality-Oriented Convolutional Filter Pruning
• HRank: Filter Pruning using High-Rank Feature Map
• Deep Learning Training on the Edge with Low-Precision Posits
• Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression
• On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks
• Network Pruning using Adaptive Exemplar Filters
• Design of Efficient Deep Learning models for Determining Road Surface Condition from Roadside Camera Images and Weather Data
• Learning Class Unique Features in Fine-Grained Visual Classification
• DeepEverest: Accelerating Declarative Top-K Queries for Deep Neural Network Interpretation
• Bespoke vs. Prêt-à-Porter Lottery Tickets: Exploiting Mask Similarity for Trainable Sub-Network Finding
• Deep Residual Networks Learn the Geodesic Curve in the Wasserstein Space
• Can the state of relevant neurons in a deep neural networks serve as indicators for detecting adversarial attacks?
• Implicit Filter Sparsification In Convolutional Neural Networks
• BEAN: Interpretable Representation Learning with Biologically-Enhanced Artificial Neuronal Assembly Regularization
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
• Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units
• Canoe : A System for Collaborative Learning for Neural Nets
• Are there any 'object detectors' in the hidden layers of CNNs trained to identify objects or scenes?