Regularizing towards Causal Invariance: Linear Models with Proxies
arXiv:2103.02477
Abstract
We propose a method for learning linear models whose predictive performance is robust to causal interventions on unobserved variables, when noisy proxies of those variables are available. Our approach takes the form of a regularization term that trades off between in-distribution performance and robustness to interventions. Under the assumption of a linear structural causal model, we show that a single proxy can be used to create estimators that are prediction optimal under interventions of bounded strength. This strength depends on the magnitude of the measurement noise in the proxy, which is, in general, not identifiable. In the case of two proxy variables, we propose a modified estimator that is prediction optimal under interventions up to a known strength. We further show how to extend these estimators to scenarios where additional information about the "test time" intervention is available during training. We evaluate our theoretical findings in synthetic experiments and using real data of hourly pollution levels across several cities in China.
arXiv commentICML 2021 (to appear)
References in corpus (13)
- Invariant Risk Minimization
- Invariant Models for Causal Transfer Learning
- The Risks of Invariant Risk Minimization
- Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions
- An Introduction to Proximal Causal Learning
- A confounding bridge approach for double negative control inference on causal effects
- Robustness to Spurious Correlations via Human Annotations
- Does Invariant Risk Minimization Capture Invariance?
- Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix
- Distributional robustness of K-class estimators and the PULSE
- Accounting for Unobserved Confounding in Domain Generalization
- Risk Variance Penalization
- Deconfounding and Causal Regularization for Stability and External Validity