Algorithms and Theory for Multiple-Source Adaptation

Abstract

This work includes a number of novel contributions for the multiple-source adaptation problem. We present new normalized solutions with strong theoretical guarantees for the cross-entropy loss and other similar losses. We also provide new guarantees that hold in the case where the conditional probabilities for the source domains are distinct. Moreover, we give new algorithms for determining the distribution-weighted combination solution for the cross-entropy loss and other losses. We report the results of a series of experiments with real-world datasets. We find that our algorithm outperforms competing approaches by producing a single robust model that performs well on any target mixture distribution. Altogether, our theory, algorithms, and empirical results provide a full solution for the multiple-source adaptation problem with very practical benefits.

arXiv commentarXiv admin note: text overlap with arXiv:1711.05037

Cited by in corpus (31)

• Ensemble Distillation for Robust Model Fusion in Federated Learning
• Bridging Theory and Algorithm for Domain Adaptation
• Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain
• Memory-aware curriculum federated learning for breast cancer classification
• Generalizing to unseen domains via distribution matching
• Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey
• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Efficient Domain Generalization via Common-Specific Low-Rank Decomposition
• Moment Matching for Multi-Source Domain Adaptation
• Feature Alignment and Restoration for Domain Generalization and Adaptation
• Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Results
• Domain Aggregation Networks for Multi-Source Domain Adaptation
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• A survey on domain adaptation theory: learning bounds and theoretical guarantees
• PAC-Bayes and Domain Adaptation
• Learning to Combine: Knowledge Aggregation for Multi-Source Domain Adaptation
• Collaborative Training of Balanced Random Forests for Open Set Domain Adaptation
• Aggregating From Multiple Target-Shifted Sources
• A Discriminative Technique for Multiple-Source Adaptation
• A Theory of Multiple-Source Adaptation with Limited Target Labeled Data
• MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation
• Union-set Multi-source Model Adaptation for Semantic Segmentation
• Local Boosting for Weakly-Supervised Learning
• Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions
• Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources
• Learning Synthetic to Real Transfer for Localization and Navigational Tasks
• Universal Multi-Source Domain Adaptation
• Ensemble Multi-Source Domain Adaptation with Pseudolabels
• Transferrable Contrastive Learning for Visual Domain Adaptation
• Domain Consistency Regularization for Unsupervised Multi-source Domain Adaptive Classification
• Distributionally Robust Language Modeling