Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice

Abstract

In this paper, we study the formalism of unsupervised multi-class domain adaptation (multi-class UDA), which underlies a few recent algorithms whose learning objectives are only motivated empirically. Multi-Class Scoring Disagreement (MCSD) divergence is presented by aggregating the absolute margin violations in multi-class classification, and this proposed MCSD is able to fully characterize the relations between any pair of multi-class scoring hypotheses. By using MCSD as a measure of domain distance, we develop a new domain adaptation bound for multi-class UDA; its data-dependent, probably approximately correct bound is also developed that naturally suggests adversarial learning objectives to align conditional feature distributions across source and target domains. Consequently, an algorithmic framework of Multi-class Domain-adversarial learning Networks (McDalNets) is developed, and its different instantiations via surrogate learning objectives either coincide with or resemble a few recently popular methods, thus (partially) underscoring their practical effectiveness. Based on our identical theory for multi-class UDA, we also introduce a new algorithm of Domain-Symmetric Networks (SymmNets), which is featured by a novel adversarial strategy of domain confusion and discrimination. SymmNets affords simple extensions that work equally well under the problem settings of either closed set, partial, or open set UDA. We conduct careful empirical studies to compare different algorithms of McDalNets and our newly introduced SymmNets. Experiments verify our theoretical analysis and show the efficacy of our proposed SymmNets. In addition, we have made our implementation code publicly available.

arXiv commentCVPR extension; TPAMI camera ready version: https://ieeexplore.ieee.org/document/9253700; IEEE copyright; Codes are available at: https://github.com/YBZh/MultiClassDA

References in corpus (13)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Learning Transferable Features with Deep Adaptation Networks
• Conditional Adversarial Domain Adaptation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• VisDA: The Visual Domain Adaptation Challenge
• Domain Adaptation: Learning Bounds and Algorithms
• Beyond Sharing Weights for Deep Domain Adaptation
• Bridging Theory and Algorithm for Domain Adaptation
• Adversarial Dropout Regularization
• Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptation
• Moment Matching for Multi-Source Domain Adaptation
• Deep Multi-View Learning using Neuron-Wise Correlation-Maximizing Regularizers
• Part-Aware Fine-grained Object Categorization using Weakly Supervised Part Detection Network

Cited by in corpus (14)

• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
• Multi-source Domain Adaptation for Text-independent Forensic Speaker Recognition
• Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering
• Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey
• Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners
• Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation
• Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learning
• On Universal Black-Box Domain Adaptation
• Deep Adversarial Transition Learning using Cross-Grafted Generative Stacks
• Reducing the Covariate Shift by Mirror Samples in Cross Domain Alignment
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?
• Discriminative Domain-Invariant Adversarial Network for Deep Domain Generalization
• Cross-Corpora Spoken Language Identification with Domain Diversification and Generalization
• Deep Domain Generalization with Feature-norm Network