Enforcing Predictive Invariance across Structured Biomedical Domains
arXiv:2006.03908
Abstract
Many biochemical applications such as molecular property prediction require models to generalize beyond their training domains (environments). Moreover, natural environments in these tasks are structured, defined by complex descriptors such as molecular scaffolds or protein families. Therefore, most environments are either never seen during training, or contain only a single training example. To address these challenges, we propose a new regret minimization (RGM) algorithm and its extension for structured environments. RGM builds from invariant risk minimization (IRM) by recasting simultaneous optimality condition in terms of predictive regret, finding a representation that enables the predictor to compete against an oracle with hindsight access to held-out environments. The structured extension adaptively highlights variation due to complex environments via specialized domain perturbations. We evaluate our method on multiple applications: molecular property prediction, protein homology and stability prediction and show that RGM significantly outperforms previous state-of-the-art baselines.
References in corpus (9)
- Neural Message Passing for Quantum Chemistry
- Conditional Adversarial Domain Adaptation
- Domain Generalization via Invariant Feature Representation
- Discriminative Embeddings of Latent Variable Models for Structured Data
- Domain Generalization via Model-Agnostic Learning of Semantic Features
- Generalizing Across Domains via Cross-Gradient Training
- Invariant Risk Minimization
- Out-of-Distribution Generalization via Risk Extrapolation (REx)
- On Learning Invariant Representation for Domain Adaptation
Cited by in corpus (6)
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
- Towards a Theoretical Framework of Out-of-Distribution Generalization
- Accounting for Unobserved Confounding in Domain Generalization
- Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers
- Learning Stable Classifiers by Transferring Unstable Features