Sensitivity and Generalization in Neural Networks: an Empirical Study

Abstract

In practice it is often found that large over-parameterized neural networks generalize better than their smaller counterparts, an observation that appears to conflict with classical notions of function complexity, which typically favor smaller models. In this work, we investigate this tension between complexity and generalization through an extensive empirical exploration of two natural metrics of complexity related to sensitivity to input perturbations. Our experiments survey thousands of models with various fully-connected architectures, optimizers, and other hyper-parameters, as well as four different image classification datasets. We find that trained neural networks are more robust to input perturbations in the vicinity of the training data manifold, as measured by the norm of the input-output Jacobian of the network, and that it correlates well with generalization. We further establish that factors associated with poor generalization $-$ such as full-batch training or using random labels $-$ correspond to lower robustness, while factors associated with good generalization $-$ such as data augmentation and ReLU non-linearities $-$ give rise to more robust functions. Finally, we demonstrate how the input-output Jacobian norm can be predictive of generalization at the level of individual test points.

arXiv commentPublished as a conference paper at ICLR 2018

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• An Analysis of Deep Neural Network Models for Practical Applications
• On the Number of Linear Regions of Deep Neural Networks
• The Loss Surfaces of Multilayer Networks
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
• Deep Neural Networks as Gaussian Processes
• Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
• A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
• Exponential expressivity in deep neural networks through transient chaos
• Robust Large Margin Deep Neural Networks
• Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
• Qualitatively characterizing neural network optimization problems
• Representation Benefits of Deep Feedforward Networks
• On the number of response regions of deep feed forward networks with piece-wise linear activations
• Universal adversarial perturbations
• Adversarial Spheres
• Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms

Cited by in corpus (98)

• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks
• Fantastic Generalization Measures and Where to Find Them
• Classification of Machine Learning Frameworks for Data-Driven Thermal Fluid Models
• Interpreting Deep Learning-Based Networking Systems
• Data augmentation instead of explicit regularization
• How to train your neural ODE: the world of Jacobian and kinetic regularization
• Sensitivity Analysis of Deep Neural Networks
• A Modern Take on the Bias-Variance Tradeoff in Neural Networks
• On Warm-Starting Neural Network Training
• Learning Differential Equations that are Easy to Solve
• Stiffness: A New Perspective on Generalization in Neural Networks
• Three Mechanisms of Weight Decay Regularization
• SGD on Neural Networks Learns Functions of Increasing Complexity
• Is Generator Conditioning Causally Related to GAN Performance?
• Meta-Learning Requires Meta-Augmentation
• Robust Learning with Jacobian Regularization
• Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
• Dimensionality compression and expansion in Deep Neural Networks
• Riemannian Continuous Normalizing Flows
• On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
• Understanding Integrated Gradients with SmoothTaylor for Deep Neural Network Attribution
• Bad Global Minima Exist and SGD Can Reach Them
• Is it enough to optimize CNN architectures on ImageNet?
• What Do Neural Networks Learn When Trained With Random Labels?
• Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
• The Implicit and Explicit Regularization Effects of Dropout
• Empirical Studies on the Properties of Linear Regions in Deep Neural Networks
• A Scale Invariant Flatness Measure for Deep Network Minima
• Knowledge Consistency between Neural Networks and Beyond
• Learning Optimal Representations with the Decodable Information Bottleneck
• On Power Laws in Deep Ensembles
• Network Pruning That Matters: A Case Study on Retraining Variants
• Explaining Knowledge Distillation by Quantifying the Knowledge
• A case for new neural network smoothness constraints
• Discovering and Explaining the Representation Bottleneck of DNNs
• Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation
• Exploring Memorization in Adversarial Training
• AdaNet: A Scalable and Flexible Framework for Automatically Learning Ensembles
• Layer-wise Characterization of Latent Information Leakage in Federated Learning
• Towards Robust Deep Neural Networks
• Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach
• Jacobian Regularization for Mitigating Universal Adversarial Perturbations
• Pathological spectra of the Fisher information metric and its variants in deep neural networks
• Taxonomizing local versus global structure in neural network loss landscapes
• A Simple Cache Model for Image Recognition
• Needles in Haystacks: On Classifying Tiny Objects in Large Images
• Is SGD a Bayesian sampler? Well, almost
• On the interplay between noise and curvature and its effect on optimization and generalization
• Breaking the Activation Function Bottleneck through Adaptive Parameterization
• Implicit Rugosity Regularization via Data Augmentation
• Evaluation of Complexity Measures for Deep Learning Generalization in Medical Image Analysis
• The Nonlinearity Coefficient - Predicting Generalization in Deep Neural Networks
• Model Complexity of Deep Learning: A Survey
• Classifying the classifier: dissecting the weight space of neural networks
• Relative stability toward diffeomorphisms indicates performance in deep nets
• The Recurrent Neural Tangent Kernel
• Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance
• Neural Network Analytic Continuation for Monte Carlo: Improvement by Statistical Errors
• MetaInfoNet: Learning Task-Guided Information for Sample Reweighting
• On the relationship between class selectivity, dimensionality, and robustness
• mFI-PSO: A Flexible and Effective Method in Adversarial Image Generation for Deep Neural Networks
• Improving Confidence Estimates for Unfamiliar Examples
• Why flatness does and does not correlate with generalization for deep neural networks
• Tangent Space Separability in Feedforward Neural Networks
• A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
• Hyperplane Arrangements of Trained ConvNets Are Biased
• Blind interactive learning of modulation schemes: Multi-agent cooperation without co-design
• A unified survey of treatment effect heterogeneity modeling and uplift modeling
• Optimal Machine Intelligence at the Edge of Chaos
• Interpreting and Disentangling Feature Components of Various Complexity from DNNs
• Pre-interpolation loss behaviour in neural networks
• Stable Rank Normalization for Improved Generalization in Neural Networks and GANs
• Trap of Feature Diversity in the Learning of MLPs
• EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAM
• Generalisation in fully-connected neural networks for time series forecasting
• Towards Understanding the Effectiveness of Attention Mechanism
• Model-Aware Regularization For Learning Approaches To Inverse Problems
• Data-driven Regularization via Racecar Training for Generalizing Neural Networks
• A Neural-Symbolic Framework for Mental Simulation
• Analytic expressions for the output evolution of a deep neural network
• Relative Flatness and Generalization
• Mining Domain Knowledge: Improved Framework towards Automatically Standardizing Anatomical Structure Nomenclature in Radiotherapy
• Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
• Locally Linear Region Knowledge Distillation
• Recent Advances in Large Margin Learning
• Noisy Feature Mixup
• Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets
• Noise Stability Regularization for Improving BERT Fine-tuning
• What classifiers know what they don't?
• Gradient representations in ReLU networks as similarity functions
• DebiNet: Debiasing Linear Models with Nonlinear Overparameterized Neural Networks
• Inherent Noise in Gradient Based Methods
• Scientific Calculator for Designing Trojan Detectors in Neural Networks
• Tangent Space Sensitivity and Distribution of Linear Regions in ReLU Networks