Accounting for Unobserved Confounding in Domain Generalization
arXiv:2007.10653
Abstract
This paper investigates the problem of learning robust, generalizable prediction models from a combination of multiple datasets and qualitative assumptions about the underlying data-generating model. Part of the challenge of learning robust models lies in the influence of unobserved confounders that void many of the invariances and principles of minimum error presently used for this problem. Our approach is to define a different invariance property of causal solutions in the presence of unobserved confounders which, through a relaxation of this invariance, can be connected with an explicit distributionally robust optimization problem over a set of affine combination of data distributions. Concretely, our objective takes the form of a standard loss, plus a regularization term that encourages partial equality of error derivatives with respect to model parameters. We demonstrate the empirical performance of our approach on healthcare data from different modalities, including image, speech and tabular data.
References in corpus (18)
- Confounding variables can degrade generalization performance of radiological deep learning models
- Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
- Generalizing Across Domains via Cross-Gradient Training
- Certifying Some Distributional Robustness with Principled Adversarial Training
- Invariant Risk Minimization
- Out-of-Distribution Generalization via Risk Extrapolation (REx)
- Distributionally Robust Logistic Regression
- Improve Unsupervised Domain Adaptation with Mixup Training
- In Search of Lost Domain Generalization
- Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
- Learning explanations that are hard to vary
- Anchor regression: heterogeneous data meets causality
- Nonlinear Invariant Risk Minimization: A Causal Approach
- Gradient Matching for Domain Generalization
- Enforcing Predictive Invariance across Structured Biomedical Domains
- It's easy to fool yourself: Case studies on identifying bias and confounding in bio-medical datasets
- Function Norms and Regularization in Deep Networks
- Identifying Invariant Factors Across Multiple Environments with KL Regression