Bridging Theory and Algorithm for Domain Adaptation

Abstract

This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.

Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019

References in corpus (8)

• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Conditional Adversarial Domain Adaptation
• VisDA: The Visual Domain Adaptation Challenge
• Domain Adaptation: Learning Bounds and Algorithms
• Algorithms and Theory for Multiple-Source Adaptation
• Multiple Source Adaptation and the Renyi Divergence
• Adaptation Algorithm and Theory Based on Generalized Discrepancy

Cited by in corpus (73)

• Universal Domain Adaptation through Self Supervision
• Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation
• Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice
• Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
• Generalized Domain Conditioned Adaptation Network
• Cycle Self-Training for Domain Adaptation
• Learning Transferable Parameters for Unsupervised Domain Adaptation
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• Relation Matters: Foreground-aware Graph-based Relational Reasoning for Domain Adaptive Object Detection
• Few-shot Domain Adaptation by Causal Mechanism Transfer
• Privacy-preserving Federated Adversarial Domain Adaption over Feature Groups for Interpretability
• Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
• Conditional Generation of Medical Images via Disentangled Adversarial Inference
• Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay
• Decoupled Adaptation for Cross-Domain Object Detection
• Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations
• Robust Local Preserving and Global Aligning Network for Adversarial Domain Adaptation
• Minimum Class Confusion for Versatile Domain Adaptation
• Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
• Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering
• Transferable Calibration with Lower Bias and Variance in Domain Adaptation
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• Prototypical Partial Optimal Transport for Universal Domain Adaptation
• Gradually Vanishing Bridge for Adversarial Domain Adaptation
• Cross-domain Contrastive Learning for Unsupervised Domain Adaptation
• Meta-Learning across Meta-Tasks for Few-Shot Learning
• Discrepancy-Based Active Learning for Domain Adaptation
• Domain Invariant Representation Learning with Domain Density Transformations
• Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Learning from a Complementary-label Source Domain: Theory and Algorithms
• Towards Stable and Comprehensive Domain Alignment: Max-Margin Domain-Adversarial Training
• Casting a BAIT for Offline and Online Source-free Domain Adaptation
• Dual Adversarial Domain Adaptation
• Geometric Understanding of Discriminability and Transferability for Visual Domain Adaptation
• Generation, augmentation, and alignment: A pseudo-source domain based method for source-free domain adaptation
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
• Open Set Domain Adaptation: Theoretical Bound and Algorithm
• A Theory of Label Propagation for Subpopulation Shift
• Information-theoretic analysis for transfer learning
• When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights
• Delving into the Continuous Domain Adaptation
• Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation
• Camera Adaptation for Fundus-Image-Based CVD Risk Estimation
• Reducing the Covariate Shift by Mirror Samples in Cross Domain Alignment
• Discriminative Cross-Domain Feature Learning for Partial Domain Adaptation
• Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
• Self-training Avoids Using Spurious Features Under Domain Shift
• On Deep Domain Adaptation: Some Theoretical Understandings
• Learning to Transfer for Traffic Forecasting via Multi-task Learning
• Domain Conditioned Adaptation Network
• A General Upper Bound for Unsupervised Domain Adaptation
• Transferable Time-Series Forecasting under Causal Conditional Shift
• MDD-UNet: Domain Adaptation for Medical Image Segmentation with Theoretical Guarantees, a Proof of Concept
• Towards Accurate and Robust Domain Adaptation under Noisy Environments
• Adversarial Robustness for Unsupervised Domain Adaptation
• Unsupervised Domain Adaptation across FMCW Radar Configurations Using Margin Disparity Discrepancy
• Regressive Domain Adaptation for Unsupervised Keypoint Detection
• Unveiling Class-Labeling Structure for Universal Domain Adaptation
• On Label Shift in Domain Adaptation via Wasserstein Distance
• On the Theoretical Equivalence of Several Trade-Off Curves Assessing Statistical Proximity
• FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation
• Metric-Learning-Assisted Domain Adaptation
• Against Adversarial Learning: Naturally Distinguish Known and Unknown in Open Set Domain Adaptation
• Matching Embeddings for Domain Adaptation
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?
• Mixed Set Domain Adaptation
• Towards Fair Cross-Domain Adaptation via Generative Learning
• KL Guided Domain Adaptation
• Task Guided Compositional Representation Learning for ZDA
• Hard Class Rectification for Domain Adaptation
• Domain-Specific Bias Filtering for Single Labeled Domain Generalization
• Universal Multi-Source Domain Adaptation