Respecting Domain Relations: Hypothesis Invariance for Domain Generalization
arXiv:2010.07591
Abstract
In domain generalization, multiple labeled non-independent and non-identically distributed source domains are available during training while neither the data nor the labels of target domains are. Currently, learning so-called domain invariant representations (DIRs) is the prevalent approach to domain generalization. In this work, we define DIRs employed by existing works in probabilistic terms and show that by learning DIRs, overly strict requirements are imposed concerning the invariance. Particularly, DIRs aim to perfectly align representations of different domains, i.e. their input distributions. This is, however, not necessary for good generalization to a target domain and may even dispose of valuable classification information. We propose to learn so-called hypothesis invariant representations (HIRs), which relax the invariance assumptions by merely aligning posteriors, instead of aligning representations. We report experimental results on public domain generalization datasets to show that learning HIRs is more effective than learning DIRs. In fact, our approach can even compete with approaches using prior knowledge about domains.
arXiv commentaccepted at ICPR 2020
References in corpus (9)
- Frustratingly Easy Domain Adaptation
- A review of domain adaptation without target labels
- Domain Generalization via Invariant Feature Representation
- DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation
- Generalizing Across Domains via Cross-Gradient Training
- Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations
- DIVA: Domain Invariant Variational Autoencoders
- Domain Generalization via Multidomain Discriminant Analysis
- Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization