Generalizing Across Domains via Cross-Gradient Training

Abstract

We present CROSSGRAD, a method to use multi-domain training data to learn a classifier that generalizes to new domains. CROSSGRAD does not need an adaptation phase via labeled or unlabeled data, or domain features in the new domain. Most existing domain adaptation methods attempt to erase domain signals using techniques like domain adversarial training. In contrast, CROSSGRAD is free to use domain signals for predicting labels, if it can prevent overfitting on training domains. We conceptualize the task in a Bayesian setting, in which a sampling step is implemented as data augmentation, based on domain-guided perturbations of input instances. CROSSGRAD parallelly trains a label and a domain classifier on examples perturbed by loss gradients of each other's objectives. This enables us to directly perturb inputs, without separating and re-mixing domain signals while making various distributional assumptions. Empirical evaluation on three different applications where this setting is natural establishes that (1) domain-guided perturbation provides consistently better generalization to unseen domains, compared to generic instance perturbation methods, and that (2) data augmentation is a more stable and accurate method than domain adversarial training.

The first two authors contributed equally; Accepted at ICLR 2018

References in corpus (7)

• Intriguing properties of neural networks
• Frustratingly Easy Domain Adaptation
• Domain Generalization via Invariant Feature Representation
• Adversarial Discriminative Domain Adaptation
• Domain Generalization for Object Recognition with Multi-task Autoencoders
• From A to Z: Supervised Transfer of Style and Content Using Deep Neural Network Generators
• Zero-Shot Deep Domain Adaptation

Cited by in corpus (66)

• Towards Out-Of-Distribution Generalization: A Survey
• Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation
• Domain Generalization by Marginal Transfer Learning
• Heterogeneous Domain Generalization via Domain Mixup
• Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
• Generalizing to unseen domains via distribution matching
• DIVA: Domain Invariant Variational Autoencoders
• In Search of Lost Domain Generalization
• Domain Generalization using Causal Matching
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space
• Feature Alignment and Restoration for Domain Generalization and Adaptation
• Optimizing Performance of Federated Person Re-identification: Benchmarking and Analysis
• FSDR: Frequency Space Domain Randomization for Domain Generalization
• Episodic Training for Domain Generalization
• Learning to Generate Novel Domains for Domain Generalization
• Selecting Data Augmentation for Simulating Interventions
• Deep Stable Learning for Out-Of-Distribution Generalization
• Style Normalization and Restitution for Generalizable Person Re-identification
• Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization
• Towards adversarial learning of speaker-invariant representation for speech emotion recognition
• Zero-shot Domain Adaptation without Domain Semantic Descriptors
• Semi-Supervised Domain Generalization with Stochastic StyleMatch
• MixStyle Neural Networks for Domain Generalization and Adaptation
• Variational Disentanglement for Domain Generalization
• Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representations
• Learning to Learn with Variational Information Bottleneck for Domain Generalization
• In Search of netUnicorn: A Data-Collection Platform to Develop Generalizable ML Models for Network Security Problems
• Domain Invariant Representation Learning with Domain Density Transformations
• Latent Causal Invariant Model
• Enforcing Predictive Invariance across Structured Biomedical Domains
• Learning to Learn Single Domain Generalization
• Style Normalization and Restitution for Domain Generalization and Adaptation
• Respecting Domain Relations: Hypothesis Invariance for Domain Generalization
• Deep Domain-Adversarial Image Generation for Domain Generalisation
• Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Time
• Domain Generalization via Semi-supervised Meta Learning
• Accounting for Unobserved Confounding in Domain Generalization
• Model-Based Domain Generalization
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty
• Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
• Improving Confidence Estimates for Unfamiliar Examples
• Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Data
• SWAD: Domain Generalization by Seeking Flat Minima
• Learning to Diversify for Single Domain Generalization
• Domain Generalization via Inference-time Label-Preserving Target Projections
• S2R-DepthNet: Learning a Generalizable Depth-specific Structural Representation
• Focus on the Common Good: Group Distributional Robustness Follows
• CNN Feature Map Augmentation for Single-Source Domain Generalization
• Multimodal Representation Learning using Deep Multiset Canonical Correlation
• Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains
• Generalized Multi-view Shared Subspace Learning using View Bootstrapping
• Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation
• VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
• Collaborative Semantic Aggregation and Calibration for Federated Domain Generalization
• Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference
• Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint
• A Domain Generalization Perspective on Listwise Context Modeling
• Confounder Identification-free Causal Visual Feature Learning
• Towards Data-Free Domain Generalization
• Semi-Supervised Domain Generalization with Evolving Intermediate Domain
• A Novel Technique for Evidence based Conditional Inference in Deep Neural Networks via Latent Feature Perturbation
• Domain Generalization via Universal Non-volume Preserving Models
• Discriminative Domain-Invariant Adversarial Network for Deep Domain Generalization
• Discovering Spatial Relationships by Transformers for Domain Generalization