Domain Generalization by Marginal Transfer Learning

Abstract

In the problem of domain generalization (DG), there are labeled training data sets from several related prediction problems, and the goal is to make accurate predictions on future unlabeled data sets that are not known to the learner. This problem arises in several applications where data distributions fluctuate because of environmental, technical, or other sources of variation. We introduce a formal framework for DG, and argue that it can be viewed as a kind of supervised learning problem by augmenting the original feature space with the marginal distribution of feature vectors. While our framework has several connections to conventional analysis of supervised learning algorithms, several unique aspects of DG require new methods of analysis. This work lays the learning theoretic foundations of domain generalization, building on our earlier conference paper where the problem of DG was introduced (Blanchard et al., 2011). We present two formal models of data generation, corresponding notions of risk, and distribution-free generalization error analysis. By focusing our attention on kernel methods, we also provide more quantitative results and a universally consistent algorithm. An efficient implementation is provided for this algorithm, which is experimentally compared to a pooling strategy on one synthetic and three real-world data sets.

Accepted to Journal of Machine Learning Research

References in corpus (7)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Domain Generalization via Invariant Feature Representation
• Generalizing Across Domains via Cross-Gradient Training
• Learning Robust Representations by Projecting Superficial Statistics Out
• Regularized Learning for Domain Adaptation under Label Shifts
• Analysis of Kernel Mean Matching under Covariate Shift
• Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution Matching

Cited by in corpus (22)

• Domain Generalization: A Survey
• Towards Out-Of-Distribution Generalization: A Survey
• In Search of Lost Domain Generalization
• Rethinking Domain Generalization: Discriminability and Generalizability
• SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalization
• Hyperparameter Learning via Distributional Transfer
• MONK -- Outlier-Robust Mean Embedding Estimation by Median-of-Means
• Multiple Domain Experts Collaborative Learning: Multi-Source Domain Generalization For Person Re-Identification
• POEM: Polarization of Embeddings for Domain-Invariant Representations
• Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalization
• Model-Based Domain Generalization
• Picking Up Quantization Steps for Compressed Image Classification
• Domain Generalization by Functional Regression
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Stable Prediction via Leveraging Seed Variable
• Barycentric-alignment and reconstruction loss minimization for domain generalization
• Offline RL With Resource Constrained Online Deployment
• Kernel Hypothesis Testing with Set-valued Data
• Explainability-aided Domain Generalization for Image Classification
• Kernel Embedding Approaches to Orbit Determination of Spacecraft Clusters
• Discriminative Domain-Invariant Adversarial Network for Deep Domain Generalization
• Balance-Subsampled Stable Prediction