Identifying Invariant Factors Across Multiple Environments with KL Regression
arXiv:2002.08341
Abstract
Many datasets are collected from multiple environments (e.g. different labs, perturbations, etc.), and it is often advantageous to learn models and relations that are invariant across environments. Invariance can improve robustness to unknown confounders and improve generalization to new domains. We develop a novel framework -- KL regression -- to reliably estimate regression coefficients in a challenging multi-environment setting, where latent confounders affect the data from each environment. KL regression is based on a new objective of simultaneously minimizing the KL- divergence between a parametric model and the observed data from each environment. We prove that KL regression recovers the true invariant factors under a flexible confounding setup. Moreover, it is computationally efficient as we derive an analytic solution for its global optimum. In systematic experiments, we validate the improved performance of KL regression compared to commonly used approaches.
References in corpus (8)
- Scikit-learn: Machine Learning in Python
- Certifying Some Distributional Robustness with Principled Adversarial Training
- Invariant Risk Minimization
- Characterization and Greedy Learning of Interventional Markov Equivalence Classes of Directed Acyclic Graphs
- Invariant Models for Causal Transfer Learning
- Causal Discovery from Changes
- Anchor regression: heterogeneous data meets causality
- backShift: Learning causal cyclic graphs from unknown shift interventions