Distributional Anchor Regression
arXiv:2101.08224 · doi:10.1007/s11222-022-10097-z
Abstract
Prediction models often fail if train and test data do not stem from the same distribution. Out-of-distribution (OOD) generalization to unseen, perturbed test data is a desirable but difficult-to-achieve property for prediction models and in general requires strong assumptions on the data generating process (DGP). In a causally inspired perspective on OOD generalization, the test data arise from a specific class of interventions on exogenous random variables of the DGP, called anchors. Anchor regression models, introduced by Rothenhaeusler et al. (2021), protect against distributional shifts in the test data by employing causal regularization. However, so far anchor regression has only been used with a squared-error loss which is inapplicable to common responses such as censored continuous or ordinal data. Here, we propose a distributional version of anchor regression which generalizes the method to potentially censored responses with at least an ordered sample space. To this end, we combine a flexible class of parametric transformation models for distributional regression with an appropriate causal regularizer under a more general notion of residuals. In an exemplary application and several simulation scenarios we demonstrate the extent to which OOD generalization is possible.
arXiv commentUpdated preprint. Under review. 29 pages, 13 figures, 2 tables
References in corpus (10)
- Invariant Risk Minimization
- Invariant Models for Causal Transfer Learning
- CVXR: An R Package for Disciplined Convex Optimization
- Most Likely Transformations
- Conditional Transformation Models
- Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
- Anchor regression: heterogeneous data meets causality
- Representation Learning via Invariant Causal Mechanisms
- A survey on domain adaptation theory: learning bounds and theoretical guarantees
- Domain adaptation under structural causal models