Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization

Abstract

Overparameterized neural networks can be highly accurate on average on an i.i.d. test set yet consistently fail on atypical groups of the data (e.g., by learning spurious correlations that hold on average but not in such groups). Distributionally robust optimization (DRO) allows us to learn models that instead minimize the worst-case training loss over a set of pre-defined groups. However, we find that naively applying group DRO to overparameterized neural networks fails: these models can perfectly fit the training data, and any model with vanishing average training loss also already has vanishing worst-case training loss. Instead, the poor worst-case performance arises from poor generalization on some groups. By coupling group DRO models with increased regularization---a stronger-than-typical L2 penalty or early stopping---we achieve substantially higher worst-group accuracies, with 10-40 percentage point improvements on a natural language inference task and two image tasks, while maintaining high average accuracies. Our results suggest that regularization is important for worst-group generalization in the overparameterized regime, even if it is not needed for average generalization. Finally, we introduce a stochastic optimization algorithm, with convergence guarantees, to efficiently train group DRO models.

References in corpus (10)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• A systematic study of the class imbalance problem in convolutional neural networks
• Equality of Opportunity in Supervised Learning
• Invariant Risk Minimization
• Empirical Bernstein Bounds and Sample Variance Penalization
• Fairness Without Demographics in Repeated Loss Minimization
• Distributionally Robust Logistic Regression
• Does Distributionally Robust Supervised Learning Give Robust Classifiers?
• Anchor regression: heterogeneous data meets causality
• Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness

Cited by in corpus (103)

• On the Opportunities and Risks of Foundation Models
• Domain Generalization: A Survey
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Towards Out-Of-Distribution Generalization: A Survey
• Causal Attention for Interpretable and Generalizable Graph Classification
• Measuring Robustness to Natural Distribution Shifts in Image Classification
• MEMO: Test Time Robustness via Adaptation and Augmentation
• In Search of Lost Domain Generalization
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations
• Domain Generalization using Causal Matching
• Noise or Signal: The Role of Image Backgrounds in Object Recognition
• Learning from Failure: Training Debiased Classifier from Biased Classifier
• Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
• Understanding the Failure Modes of Out-of-Distribution Generalization
• Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization
• Learning Debiased Representation via Disentangled Feature Augmentation
• Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
• Robust Validation: Confident Predictions Even When Distributions Shift
• Discovering interpretable elastoplasticity models via the neural polynomial method enabled symbolic regressions
• Gradient Starvation: A Learning Proclivity in Neural Networks
• Learning perturbation sets for robust machine learning
• Large-Scale Methods for Distributionally Robust Optimization
• FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
• Enhanced Cross-Dataset Electroencephalogram-based Emotion Recognition using Unsupervised Domain Adaptation
• Towards out of distribution generalization for problems in mechanics
• Nonlinear Invariant Risk Minimization: A Causal Approach
• ScatterShot: Interactive In-context Example Curation for Text Transformation
• SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalization
• Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity
• BREEDS: Benchmarks for Subpopulation Shift
• DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems
• Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models
• Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
• Debiasing Sequential Recommenders through Distributionally Robust Optimization over System Exposure
• Gradient Matching for Domain Generalization
• POEM: Polarization of Embeddings for Domain-Invariant Representations
• Label-Imbalanced and Group-Sensitive Classification under Overparameterization
• On Calibration and Out-of-domain Generalization
• Environment Inference for Invariant Learning
• Simple data balancing achieves competitive worst-group-accuracy
• Modeling Techniques for Machine Learning Fairness: A Survey
• An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
• Kernelized Heterogeneous Risk Minimization
• Latent Causal Invariant Model
• Causally motivated Shortcut Removal Using Auxiliary Labels
• Unified Group Fairness on Federated Learning
• A Theory of Label Propagation for Subpopulation Shift
• Adaptive Domain Generalization via Online Disagreement Minimization
• Identifying Spurious Correlations for Robust Text Classification
• Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization
• Distributionally Robust Deep Learning using Hardness Weighted Sampling
• DORO: Distributional and Outlier Robust Optimization
• Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
• Coping with Label Shift via Distributionally Robust Optimisation
• Model-Based Domain Generalization
• Domain Generalization for Session-Independent Brain-Computer Interface
• Accounting for Unobserved Confounding in Domain Generalization
• An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization
• Teacher's pet: understanding and mitigating biases in distillation
• Environment Invariant Linear Least Squares
• Learning Debiased and Disentangled Representations for Semantic Segmentation
• Risk Variance Penalization
• Selective Classification Can Magnify Disparities Across Groups
• Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domain
• A Fine-Grained Analysis on Distribution Shift
• Towards Principled Disentanglement for Domain Generalization
• Learning Representations that Support Robust Transfer of Predictors
• BiaSwap: Removing dataset bias with bias-tailored swapping augmentation
• Robustness through Data Augmentation Loss Consistency
• Modeling the Second Player in Distributionally Robust Optimization
• Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces
• Why do classifier accuracies show linear trends under distribution shift?
• Understanding the effect of sparsity on neural networks robustness
• Optimization-based Causal Estimation from Heterogenous Environments
• Reducing Bias in Pre-trained Models by Tuning while Penalizing Change
• Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers
• Learning Stable Classifiers by Transferring Unstable Features
• Robust Representation Learning via Perceptual Similarity Metrics
• Exploring the Unfairness of DP-SGD Across Settings
• Investigating Group Distributionally Robust Optimization for Deep Imbalanced Learning: A Case Study of Binary Tabular Data Classification
• On Language Models for Creoles
• Enhancing Model Robustness and Fairness with Causality: A Regularization Approach
• A call for better unit testing for invariant risk minimisation
• Out-of-Distribution Generalization in Kernel Regression
• Explainability-aided Domain Generalization for Image Classification
• Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
• Unsupervised Learning of Debiased Representations with Pseudo-Attributes
• Generalizing Nucleus Recognition Model in Multi-source Images via Pruning
• Reappraising Domain Generalization in Neural Networks
• Combining Diverse Feature Priors
• Are Compressed Language Models Less Subgroup Robust?
• An Empirical Framework for Domain Generalization in Clinical Settings
• Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation
• Procrustean Training for Imbalanced Deep Learning
• Active Assessment of Prediction Services as Accuracy Surface Over Attribute Combinations
• Distributionally Robust Learning with Stable Adversarial Training
• Distributionally Robust Multilingual Machine Translation
• Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates
• Generalization by Recognizing Confusion
• Balancing Average and Worst-case Accuracy in Multitask Learning
• Boosted CVaR Classification
• Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles