Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift

Abstract

Adversarial learning has demonstrated good performance in the unsupervised domain adaptation setting, by learning domain-invariant representations. However, recent work has shown limitations of this approach when label distributions differ between the source and target domains. In this paper, we propose a new assumption, generalized label shift ($GLS$), to improve robustness against mismatched label distributions. $GLS$ states that, conditioned on the label, there exists a representation of the input that is invariant between the source and target domains. Under $GLS$, we provide theoretical guarantees on the transfer performance of any classifier. We also devise necessary and sufficient conditions for $GLS$ to hold, by using an estimation of the relative class weights between domains and an appropriate reweighting of samples. Our weight estimation method could be straightforwardly and generically applied in existing domain adaptation (DA) algorithms that learn domain-invariant representations, with small computational overhead. In particular, we modify three DA algorithms, JAN, DANN and CDAN, and evaluate their performance on standard and artificial DA tasks. Our algorithms outperform the base versions, with vast improvements for large label distribution mismatches. Our code is available at https://tinyurl.com/y585xt6j.

Appeared in NeurIPS 2020

References in corpus (12)

• How transferable are features in deep neural networks?
• Learning Transferable Features with Deep Adaptation Networks
• Frustratingly Easy Domain Adaptation
• Learning Representations by Maximizing Mutual Information Across Views
• Domain Separation Networks
• VisDA: The Visual Domain Adaptation Challenge
• Adversarial Discriminative Domain Adaptation
• Properties of Classical and Quantum Jensen-Shannon Divergence
• On Learning Invariant Representation for Domain Adaptation
• Domain Agnostic Learning with Disentangled Representations
• On Causal and Anticausal Learning
• Deep Transfer Network: Unsupervised Domain Adaptation

Cited by in corpus (16)

• Rethinking Distributional Matching Based Domain Adaptation
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation
• Domain Invariant Representation Learning with Domain Density Transformations
• Aggregating From Multiple Target-Shifted Sources
• When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights
• Understanding the Limits of Unsupervised Domain Adaptation via Data Poisoning
• Cross-Domain Sentiment Classification with In-Domain Contrastive Learning
• Factorizable Joint Shift in Multinomial Classification
• Learning Invariant Representation with Consistency and Diversity for Semi-supervised Source Hypothesis Transfer
• Target Consistency for Domain Adaptation: when Robustness meets Transferability
• FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation
• Cross-Domain Sentiment Classification with Contrastive Learning and Mutual Information Maximization
• KL Guided Domain Adaptation
• ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation