Joint Contrastive Learning for Unsupervised Domain Adaptation

Abstract

Enhancing feature transferability by matching marginal distributions has led to improvements in domain adaptation, although this is at the expense of feature discrimination. In particular, the ideal joint hypothesis error in the target error upper bound, which was previously considered to be minute, has been found to be significant, impairing its theoretical guarantee. In this paper, we propose an alternative upper bound on the target error that explicitly considers the joint error to render it more manageable. With the theoretical analysis, we suggest a joint optimization framework that combines the source and target domains. Further, we introduce Joint Contrastive Learning (JCL) to find class-level discriminative features, which is essential for minimizing the joint error. With a solid theoretical framework, JCL employs contrastive loss to maximize the mutual information between a feature and its label, which is equivalent to maximizing the Jensen-Shannon divergence between conditional distributions. Experiments on two real-world datasets demonstrate that JCL outperforms the state-of-the-art methods.

16 pages, 1 figure, 4 tables

References in corpus (13)

• A Simple Framework for Contrastive Learning of Visual Representations
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Learning deep representations by mutual information estimation and maximization
• Conditional Adversarial Domain Adaptation
• Momentum Contrast for Unsupervised Visual Representation Learning
• VisDA: The Visual Domain Adaptation Challenge
• Contrastive Multiview Coding
• Asymmetric Tri-training for Unsupervised Domain Adaptation
• On Variational Bounds of Mutual Information
• Contrastively Smoothed Class Alignment for Unsupervised Domain Adaptation
• Joint Semantic Domain Alignment and Target Classifier Learning for Unsupervised Domain Adaptation
• Learning Condensed and Aligned Features for Unsupervised Domain Adaptation Using Label Propagation

Cited by in corpus (5)

• Review-Based Domain Disentanglement without Duplicate Users or Contexts for Cross-Domain Recommendation
• Cross-domain Contrastive Learning for Unsupervised Domain Adaptation
• Gradient Regularized Contrastive Learning for Continual Domain Adaptation
• Unsupervised domain adaptation via coarse-to-fine feature alignment method using contrastive learning
• Prototypical Cross-domain Knowledge Transfer for Cervical Dysplasia Visual Inspection