Generalizing to unseen domains via distribution matching

Abstract

Supervised learning results typically rely on assumptions of i.i.d. data. Unfortunately, those assumptions are commonly violated in practice. In this work, we tackle such problem by focusing on domain generalization: a formalization where the data generating process at test time may yield samples from never-before-seen domains (distributions). Our work relies on the following lemma: by minimizing a notion of discrepancy between all pairs from a set of given domains, we also minimize the discrepancy between any pairs of mixtures of domains. Using this result, we derive a generalization bound for our setting. We then show that low risk over unseen domains can be achieved by representing the data in a space where (i) the training distributions are indistinguishable, and (ii) relevant information for the task at hand is preserved. Minimizing the terms in our bound yields an adversarial formulation which estimates and minimizes pairwise discrepancies. We validate our proposed strategy on standard domain generalization benchmarks, outperforming a number of recently introduced methods. Notably, we tackle a real-world application where the underlying data corresponds to multi-channel electroencephalography time series from different subjects, each considered as a distinct domain.

This work has been submitted to the IEEE for possible publication

References in corpus (16)

• Learning Transferable Features with Deep Adaptation Networks
• Domain Generalization via Invariant Feature Representation
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Generalizing Across Domains via Cross-Gradient Training
• Invariant Risk Minimization
• On Learning Invariant Representation for Domain Adaptation
• Algorithms and Theory for Multiple-Source Adaptation
• Domain Generalization using Causal Matching
• Stabilizing GAN Training with Multiple Random Projections
• Invariant Representations for Noisy Speech Recognition
• Invariant Risk Minimization Games
• Episodic Training for Domain Generalization
• Improving out-of-distribution generalization via multi-task self-supervised pretraining
• Multi-objective training of Generative Adversarial Networks with multiple discriminators
• Multi-Domain Adversarial Learning

Cited by in corpus (22)

• Domain Generalization: A Survey
• Out-of-Distribution Generalization via Risk Extrapolation (REx)
• Domain Generalization using Causal Matching
• Understanding the Failure Modes of Out-of-Distribution Generalization
• Federated Learning with Domain Generalization
• AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
• Empirical or Invariant Risk Minimization? A Sample Complexity Perspective
• Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization
• Model-Based Domain Generalization
• An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization
• Quantifying and Improving Transferability in Domain Generalization
• Towards Principled Disentanglement for Domain Generalization
• Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments
• Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
• Barycentric-alignment and reconstruction loss minimization for domain generalization
• Explainability-aided Domain Generalization for Image Classification
• Discrepancy Minimization in Domain Generalization with Generative Nearest Neighbors
• Generalizing Nucleus Recognition Model in Multi-source Images via Pruning
• Robust White Matter Hyperintensity Segmentation on Unseen Domain
• Domain Conditional Predictors for Domain Adaptation
• Improved Detection of Face Presentation Attacks Using Image Decomposition
• Robust Domain-Free Domain Generalization with Class-aware Alignment