Bridging Theory and Algorithm for Domain Adaptation

Abstract

This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.

Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019

References in corpus (5)

• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• VisDA: The Visual Domain Adaptation Challenge
• Multiple Source Adaptation and the Renyi Divergence
• Adaptation Algorithm and Theory Based on Generalized Discrepancy

Cited by in corpus (14)

• Generalized Domain Conditioned Adaptation Network
• Dual Adversarial Domain Adaptation
• Generation, augmentation, and alignment: A pseudo-source domain based method for source-free domain adaptation
• Information-theoretic analysis for transfer learning
• Discriminative Cross-Domain Feature Learning for Partial Domain Adaptation
• Domain Conditioned Adaptation Network
• A General Upper Bound for Unsupervised Domain Adaptation
• Adversarial Robustness for Unsupervised Domain Adaptation
• Towards Accurate and Robust Domain Adaptation under Noisy Environments
• Against Adversarial Learning: Naturally Distinguish Known and Unknown in Open Set Domain Adaptation
• Mixed Set Domain Adaptation
• Task Guided Compositional Representation Learning for ZDA
• Universal Multi-Source Domain Adaptation
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?