Environment Inference for Invariant Learning

Abstract

Learning models that gracefully handle distribution shifts is central to research on domain generalization, robust optimization, and fairness. A promising formulation is domain-invariant learning, which identifies the key issue of learning which features are domain-specific versus domain-invariant. An important assumption in this area is that the training examples are partitioned into "domains" or "environments". Our focus is on the more common setting where such partitions are not provided. We propose EIIL, a general framework for domain-invariant learning that incorporates Environment Inference to directly infer partitions that are maximally informative for downstream Invariant Learning. We show that EIIL outperforms invariant learning methods on the CMNIST benchmark without using environment labels, and significantly outperforms ERM on worst-group performance in the Waterbirds and CivilComments datasets. Finally, we establish connections between EIIL and algorithmic fairness, which enables EIIL to improve accuracy and calibration in a fair prediction problem.

References in corpus (19)

• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Shortcut Learning in Deep Neural Networks
• Equality of Opportunity in Supervised Learning
• Conditional Adversarial Domain Adaptation
• Do ImageNet Classifiers Generalize to ImageNet?
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Towards a Critical Race Methodology in Algorithmic Fairness
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• The Frontiers of Fairness in Machine Learning
• Fairness Without Demographics in Repeated Loss Minimization
• The Risks of Invariant Risk Minimization
• Just Train Twice: Improving Group Robustness without Training Group Information
• Learning from Failure: Training Debiased Classifier from Biased Classifier
• No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems
• The implicit fairness criterion of unconstrained learning
• Robustness to Spurious Correlations via Human Annotations
• Fairness risk measures
• Does Invariant Risk Minimization Capture Invariance?

Cited by in corpus (4)

• Fairness without Demographics through Adversarially Reweighted Learning
• Unshuffling Data for Improved Generalization
• Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix
• Kernelized Heterogeneous Risk Minimization