On the importance of single directions for generalization

Abstract

Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (defined as the activation of a single unit or some linear combination of units in response to some input) have been highlighted, but their importance has not been evaluated. Here, we connect these lines of inquiry to demonstrate that a network's reliance on single directions is a good predictor of its generalization performance, across networks trained on datasets with different fractions of corrupted labels, across ensembles of networks trained on datasets with unmodified labels, across different hyperparameters, and over the course of training. While dropout only regularizes this quantity up to a point, batch normalization implicitly discourages single direction reliance, in part by decreasing the class selectivity of individual units. Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.

arXiv commentICLR 2018 conference paper; added additional methodological details

References in corpus (4)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Opening the Black Box of Deep Neural Networks via Information
• Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
• Analyzing the Performance of Multilayer Neural Networks for Object Recognition

Cited by in corpus (73)

• Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
• Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
• Understanding the Role of Individual Units in a Deep Neural Network
• FedBN: Federated Learning on Non-IID Features via Local Batch Normalization
• Multi-task Self-Supervised Learning for Human Activity Detection
• How Does Batch Normalization Help Optimization?
• GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
• Ablation Studies in Artificial Neural Networks
• Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI
• Micro-Batch Training with Batch-Channel Normalization and Weight Standardization
• Deep learning generalizes because the parameter-function map is biased towards simple functions
• Revisiting the Importance of Individual Units in CNNs via Ablation
• Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs
• Stronger generalization bounds for deep nets via a compression approach
• How Important Is a Neuron?
• Self-supervised learning through the eyes of a child
• Video Representation Learning with Visual Tempo Consistency
• Insights on representational similarity in neural networks with canonical correlation
• On Information Plane Analyses of Neural Network Classifiers -- A Review
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
• Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets
• AI Safety for Everyone
• Identifying and Controlling Important Neurons in Neural Machine Translation
• Towards falsifiable interpretability research
• Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation
• Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics
• Understanding Generalization through Visualizations
• Distilling Critical Paths in Convolutional Neural Networks
• Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs
• Handcrafted Backdoors in Deep Neural Networks
• Learning Optimal Representations with the Decodable Information Bottleneck
• Representation Based Complexity Measures for Predicting Generalization in Deep Learning
• Auditing Data Provenance in Text-Generation Models
• Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
• Ablation of a Robot's Brain: Neural Networks Under a Knife
• Interference and Generalization in Temporal Difference Learning
• Towards Understanding Hierarchical Learning: Benefits of Neural Representations
• TIMELY: Pushing Data Movements and Interfaces in PIM Accelerators Towards Local and in Time Domain
• Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression
• Rethinking Normalization and Elimination Singularity in Neural Networks
• Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps
• Why Unsupervised Deep Networks Generalize
• The functional role of cue-driven feature-based feedback in object recognition
• Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
• Channel Equilibrium Networks for Learning Deep Representation
• On the relationship between class selectivity, dimensionality, and robustness
• Dynamic Sparse Graph for Efficient Deep Learning
• Detecting Dead Weights and Units in Neural Networks
• Implicit Gradient Regularization
• Network Pruning using Adaptive Exemplar Filters
• Abstraction Mechanisms Predict Generalization in Deep Neural Networks
• An Efficient Explorative Sampling Considering the Generative Boundaries of Deep Generative Neural Networks
• Learning Class Unique Features in Fine-Grained Visual Classification
• Detecting Memorization in ReLU Networks
• RePr: Improved Training of Convolutional Filters
• YASENN: Explaining Neural Networks via Partitioning Activation Sequences
• Interpretable Neural Network Decoupling
• Distribution of Classification Margins: Are All Data Equal?
• Batch Normalization Sampling
• BEAN: Interpretable Representation Learning with Biologically-Enhanced Artificial Neuronal Assembly Regularization
• Analyzing Visual Representations in Embodied Navigation Tasks
• Implicit Filter Sparsification In Convolutional Neural Networks
• Global Context Networks
• How Do You Act? An Empirical Study to Understand Behavior of Deep Reinforcement Learning Agents
• Understanding of Kernels in CNN Models by Suppressing Irrelevant Visual Features in Images
• The Roles of Supervised Machine Learning in Systems Neuroscience
• Gradient-Adjusted Neuron Activation Profiles for Comprehensive Introspection of Convolutional Speech Recognition Models
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
• Separating Content and Style for Unsupervised Image-to-Image Translation
• Geometric algorithms for predicting resilience and recovering damage in neural networks
• On the Pitfalls of Analyzing Individual Neurons in Language Models
• New Interpretations of Normalization Methods in Deep Learning