Gradient Starvation: A Learning Proclivity in Neural Networks

Abstract

We identify and formalize a fundamental gradient descent phenomenon resulting in a learning proclivity in over-parameterized neural networks. Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task, despite the presence of other predictive features that fail to be discovered. This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks. Using tools from Dynamical Systems theory, we identify simple properties of learning dynamics during gradient descent that lead to this imbalance, and prove that such a situation can be expected given certain statistical structure in training data. Based on our proposed formalism, we develop guarantees for a novel regularization method aimed at decoupling feature learning dynamics, improving accuracy and robustness in cases hindered by gradient starvation. We illustrate our findings with simple and real-world out-of-distribution (OOD) generalization experiments.

arXiv commentProceeding of NeurIPS 2021

References in corpus (47)

• Adam: A Method for Stochastic Optimization
• Shortcut Learning in Deep Neural Networks
• Confounding variables can degrade generalization performance of radiological deep learning models
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Understanding deep learning requires rethinking generalization
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• A Convergence Theory for Deep Learning via Over-Parameterization
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• Synthetic and Natural Noise Both Break Neural Machine Translation
• On Exact Computation with an Infinitely Wide Neural Net
• Gradient Descent Provably Optimizes Over-parameterized Neural Networks
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• A Closer Look at Memorization in Deep Networks
• Black-box Adversarial Attacks with Limited Queries and Information
• Invariant Risk Minimization
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks
• A mathematical theory of semantic development in deep neural networks
• The generalization error of random features regression: Precise asymptotics and double descent curve
• Measuring the tendency of CNNs to Learn Surface Statistical Regularities
• Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks
• On the Spectral Bias of Neural Networks
• Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
• The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies
• Deep learning generalizes because the parameter-function map is biased towards simple functions
• Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
• The Pitfalls of Simplicity Bias in Neural Networks
• Geometry of Optimization and Implicit Regularization in Deep Learning
• Learning explanations that are hard to vary
• Towards Understanding the Spectral Bias of Deep Learning
• Excessive Invariance Causes Adversarial Vulnerability
• Learning from Failure: Training Debiased Classifier from Biased Classifier
• Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks
• What shapes feature representations? Exploring datasets, architectures, and training
• An analytic theory of generalization dynamics and transfer learning in deep linear networks
• Theory of Deep Learning III: explaining the non-overfitting puzzle
• SGD on Neural Networks Learns Functions of Increasing Complexity
• Dynamics of Deep Neural Networks and Neural Tangent Hierarchy
• Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian
• Probing Neural Network Comprehension of Natural Language Arguments
• Invariant Risk Minimization Games
• Cross-Entropy Loss and Low-Rank Features Have Responsibility for Adversarial Examples
• Gradient Descent for One-Hidden-Layer Neural Networks: Polynomial Convergence and SQ Lower Bounds
• Implicit Regularization via Neural Feature Alignment
• Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions

Cited by in corpus (11)

• Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style
• Fairness via Representation Neutralization
• SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalization
• Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
• The Low-Rank Simplicity Bias in Deep Networks
• Simple data balancing achieves competitive worst-group-accuracy
• Towards Understanding the Data Dependency of Mixup-style Training
• Quantifying and Improving Transferability in Domain Generalization
• Encouraging Intra-Class Diversity Through a Reverse Contrastive Loss for Better Single-Source Domain Generalization
• Combating Unknown Bias with Effective Bias-Conflicting Scoring and Gradient Alignment
• Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization