Generalizing Across Domains via Cross-Gradient Training

Abstract

We present CROSSGRAD, a method to use multi-domain training data to learn a classifier that generalizes to new domains. CROSSGRAD does not need an adaptation phase via labeled or unlabeled data, or domain features in the new domain. Most existing domain adaptation methods attempt to erase domain signals using techniques like domain adversarial training. In contrast, CROSSGRAD is free to use domain signals for predicting labels, if it can prevent overfitting on training domains. We conceptualize the task in a Bayesian setting, in which a sampling step is implemented as data augmentation, based on domain-guided perturbations of input instances. CROSSGRAD parallelly trains a label and a domain classifier on examples perturbed by loss gradients of each other's objectives. This enables us to directly perturb inputs, without separating and re-mixing domain signals while making various distributional assumptions. Empirical evaluation on three different applications where this setting is natural establishes that (1) domain-guided perturbation provides consistently better generalization to unseen domains, compared to generic instance perturbation methods, and that (2) data augmentation is a more stable and accurate method than domain adversarial training.

The first two authors contributed equally; Accepted at ICLR 2018

References in corpus (3)

• Frustratingly Easy Domain Adaptation
• Domain Generalization via Invariant Feature Representation
• Adversarial Discriminative Domain Adaptation

Cited by in corpus (65)

• Towards Out-Of-Distribution Generalization: A Survey
• Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation
• Heterogeneous Domain Generalization via Domain Mixup
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
• Generalizing to unseen domains via distribution matching
• DIVA: Domain Invariant Variational Autoencoders
• In Search of Lost Domain Generalization
• Domain Generalization using Causal Matching
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space
• Feature Alignment and Restoration for Domain Generalization and Adaptation
• Optimizing Performance of Federated Person Re-identification: Benchmarking and Analysis
• FSDR: Frequency Space Domain Randomization for Domain Generalization
• Episodic Training for Domain Generalization
• Learning to Generate Novel Domains for Domain Generalization
• Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization
• Deep Stable Learning for Out-Of-Distribution Generalization
• Selecting Data Augmentation for Simulating Interventions
• Style Normalization and Restitution for Generalizable Person Re-identification
• Towards adversarial learning of speaker-invariant representation for speech emotion recognition
• Zero-shot Domain Adaptation without Domain Semantic Descriptors
• MixStyle Neural Networks for Domain Generalization and Adaptation
• Semi-Supervised Domain Generalization with Stochastic StyleMatch
• Variational Disentanglement for Domain Generalization
• Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representations
• Learning to Learn with Variational Information Bottleneck for Domain Generalization
• Domain Invariant Representation Learning with Domain Density Transformations
• In Search of netUnicorn: A Data-Collection Platform to Develop Generalizable ML Models for Network Security Problems
• Latent Causal Invariant Model
• Style Normalization and Restitution for Domain Generalization and Adaptation
• Respecting Domain Relations: Hypothesis Invariance for Domain Generalization
• Enforcing Predictive Invariance across Structured Biomedical Domains
• Learning to Learn Single Domain Generalization
• Deep Domain-Adversarial Image Generation for Domain Generalisation
• Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Time
• Domain Generalization via Semi-supervised Meta Learning
• Accounting for Unobserved Confounding in Domain Generalization
• Model-Based Domain Generalization
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty
• Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
• Learning to Diversify for Single Domain Generalization
• Improving Confidence Estimates for Unfamiliar Examples
• Domain Generalization via Inference-time Label-Preserving Target Projections
• SWAD: Domain Generalization by Seeking Flat Minima
• Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data
• Focus on the Common Good: Group Distributional Robustness Follows
• S2R-DepthNet: Learning a Generalizable Depth-specific Structural Representation
• CNN Feature Map Augmentation for Single-Source Domain Generalization
• Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains
• VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
• Generalized Multi-view Shared Subspace Learning using View Bootstrapping
• Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation
• Multimodal Representation Learning using Deep Multiset Canonical Correlation
• A Domain Generalization Perspective on Listwise Context Modeling
• Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint
• Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference
• Confounder Identification-free Causal Visual Feature Learning
• Collaborative Semantic Aggregation and Calibration for Federated Domain Generalization
• Domain Generalization via Universal Non-volume Preserving Models
• Semi-Supervised Domain Generalization with Evolving Intermediate Domain
• Discovering Spatial Relationships by Transformers for Domain Generalization
• Towards Data-Free Domain Generalization
• Discriminative Domain-Invariant Adversarial Network for Deep Domain Generalization
• A Novel Technique for Evidence based Conditional Inference in Deep Neural Networks via Latent Feature Perturbation