Understanding the Failure Modes of Out-of-Distribution Generalization

Abstract

Empirical studies suggest that machine learning models often rely on features, such as the background, that may be spuriously correlated with the label only during training time, resulting in poor accuracy during test-time. In this work, we identify the fundamental factors that give rise to this behavior, by explaining why models fail this way {\em even} in easy-to-learn tasks where one would expect these models to succeed. In particular, through a theoretical study of gradient-descent-trained linear classifiers on some easy-to-learn tasks, we uncover two complementary failure modes. These modes arise from how spurious correlations induce two kinds of skews in the data: one geometric in nature, and another, statistical in nature. Finally, we construct natural modifications of image classification datasets to understand when these failure modes can arise in practice. We also design experiments to isolate the two failure modes when training modern neural networks on these datasets.

References in corpus (20)

• mixup: Beyond Empirical Risk Minimization
• Shortcut Learning in Deep Neural Networks
• Domain Generalization via Invariant Feature Representation
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• A Closer Look at Memorization in Deep Networks
• Exploring Generalization in Deep Learning
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
• Uniform convergence may be unable to explain generalization in deep learning
• Generalizing to unseen domains via distribution matching
• In Search of Lost Domain Generalization
• An Investigation of Why Overparameterization Exacerbates Spurious Correlations
• Risk and parameter convergence of logistic regression
• Generalization in Deep Networks: The Role of Distance from Initialization
• What shapes feature representations? Exploring datasets, architectures, and training
• SGD on Neural Networks Learns Functions of Increasing Complexity
• Cross-Entropy Loss and Low-Rank Features Have Responsibility for Adversarial Examples
• Towards Shape Biased Unsupervised Representation Learning for Domain Generalization
• On the Learning Dynamics of Deep Neural Networks
• Feature Noise Induces Loss Discrepancy Across Groups

Cited by in corpus (16)

• On the Opportunities and Risks of Foundation Models
• Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
• Towards a Theoretical Framework of Out-of-Distribution Generalization
• Towards out of distribution generalization for problems in mechanics
• Nonlinear Invariant Risk Minimization: A Causal Approach
• Deep Learning Through the Lens of Example Difficulty
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
• Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
• Causally motivated Shortcut Removal Using Auxiliary Labels
• Dangers of Bayesian Model Averaging under Covariate Shift
• Model-Based Domain Generalization
• Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data
• Experimental Standards for Deep Learning in Natural Language Processing Research
• Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing
• I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch
• Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data