Out of Distribution Generalization in Machine Learning

Abstract

Machine learning has achieved tremendous success in a variety of domains in recent years. However, a lot of these success stories have been in places where the training and the testing distributions are extremely similar to each other. In everyday situations when models are tested in slightly different data than they were trained on, ML algorithms can fail spectacularly. This research attempts to formally define this problem, what sets of assumptions are reasonable to make in our data and what kind of guarantees we hope to obtain from them. Then, we focus on a certain class of out of distribution problems, their assumptions, and introduce simple algorithms that follow from these assumptions that are able to provide more reliable generalization. A central topic in the thesis is the strong link between discovering the causal structure of the data, finding features that are reliable (when using them to predict) regardless of their context, and out of distribution generalization.

References in corpus (12)

• Sequence to Sequence Learning with Neural Networks
• Prototypical Networks for Few-shot Learning
• Neural Architecture Search with Reinforcement Learning
• Equality of Opportunity in Supervised Learning
• Deep Learning: A Critical Appraisal
• Certifying Some Distributional Robustness with Principled Adversarial Training
• Invariant Risk Minimization
• The Numerics of GANs
• Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search
• A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
• Does Distributionally Robust Supervised Learning Give Robust Classifiers?
• Anchor regression: heterogeneous data meets causality

Cited by in corpus (7)

• Nonlinear Invariant Risk Minimization: A Causal Approach
• In Search of Deep Learning Architectures for Load Forecasting: A Comparative Analysis and the Impact of the Covid-19 Pandemic on Model Performance
• Dangers of Bayesian Model Averaging under Covariate Shift
• Optimization-based Causal Estimation from Heterogenous Environments
• Out-of-Distribution Generalization in Kernel Regression
• On Invariance Penalties for Risk Minimization
• Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers