Efficient Domain Generalization via Common-Specific Low-Rank Decomposition

Abstract

Domain generalization refers to the task of training a model which generalizes to new domains that are not seen during training. We present CSD (Common Specific Decomposition), for this setting,which jointly learns a common component (which generalizes to new domains) and a domain specific component (which overfits on training domains). The domain specific components are discarded after training and only the common component is retained. The algorithm is extremely simple and involves only modifying the final linear classification layer of any given neural network architecture. We present a principled analysis to understand existing approaches, provide identifiability results of CSD,and study effect of low-rank on domain generalization. We show that CSD either matches or beats state of the art approaches for domain generalization based on domain erasure, domain perturbed data augmentation, and meta-learning. Further diagnostics on rotated MNIST, where domains are interpretable, confirm the hypothesis that CSD successfully disentangles common and domain specific components and hence leads to better domain generalization.

References in corpus (6)

• Frustratingly Easy Domain Adaptation
• Domain Generalization via Invariant Feature Representation
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• Invariant Risk Minimization
• Algorithms and Theory for Multiple-Source Adaptation
• Learning Robust Representations by Projecting Superficial Statistics Out

Cited by in corpus (12)

• Domain Generalization using Causal Matching
• Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
• Towards out of distribution generalization for problems in mechanics
• Empirical or Invariant Risk Minimization? A Sample Complexity Perspective
• POEM: Polarization of Embeddings for Domain-Invariant Representations
• Domain Invariant Representation Learning with Domain Density Transformations
• Parameter Exchange for Robust Dynamic Domain Generalization
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty
• Domain Generalization via Inference-time Label-Preserving Target Projections
• Calibrated Feature Decomposition for Generalizable Person Re-Identification
• Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint
• An Empirical Framework for Domain Generalization in Clinical Settings