Model-Based Domain Generalization
arXiv:2102.11436
Abstract
Despite remarkable success in a variety of applications, it is well-known that deep learning can fail catastrophically when presented with out-of-distribution data. Toward addressing this challenge, we consider the domain generalization problem, wherein predictors are trained using data drawn from a family of related training domains and then evaluated on a distinct and unseen test domain. We show that under a natural model of data generation and a concomitant invariance condition, the domain generalization problem is equivalent to an infinite-dimensional constrained statistical learning problem; this problem forms the basis of our approach, which we call Model-Based Domain Generalization. Due to the inherent challenges in solving constrained optimization problems in deep learning, we exploit nonconvex duality theory to develop unconstrained relaxations of this statistical problem with tight bounds on the duality gap. Based on this theoretical motivation, we propose a novel domain generalization algorithm with convergence guarantees. In our experiments, we report improvements of up to 30 percentage points over state-of-the-art domain generalization baselines on several benchmarks including ColoredMNIST, Camelyon17-WILDS, FMoW-WILDS, and PACS.
References in corpus (30)
- Frustratingly Easy Domain Adaptation
- Domain Generalization via Invariant Feature Representation
- AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
- Domain Generalization via Model-Agnostic Learning of Semantic Features
- Domain Adaptation for Visual Applications: A Comprehensive Survey
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
- Improve Unsupervised Domain Adaptation with Mixup Training
- Heterogeneous Domain Generalization via Domain Mixup
- Learning Robust Representations by Projecting Superficial Statistics Out
- The Risks of Invariant Risk Minimization
- In Search of Lost Domain Generalization
- Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularization
- Feature-Critic Networks for Heterogeneous Domain Generalization
- DC3: A learning method for optimization with hard constraints
- Noise or Signal: The Role of Image Backgrounds in Object Recognition
- Understanding the Failure Modes of Out-of-Distribution Generalization
- Self-Challenging Improves Cross-Domain Generalization
- No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems
- When Unseen Domain Generalization is Unnecessary? Rethinking Data Augmentation
- Nonlinear Invariant Risk Minimization: A Causal Approach
- Does Invariant Risk Minimization Capture Invariance?
- Constrained Learning with Non-Convex Losses
- BREEDS: Benchmarks for Subpopulation Shift
- A Generalization Error Bound for Multi-class Domain Generalization
- Medical Image Harmonization Using Deep Learning Based Canonical Mapping: Toward Robust and Generalizable Learning in Imaging
- Empirical or Invariant Risk Minimization? A Sample Complexity Perspective
- Gradient Matching for Domain Generalization
- Robust Reinforcement Learning using Adversarial Populations
- FOCUS: Familiar Objects in Common and Uncommon Settings