Domain-Specific Batch Normalization for Unsupervised Domain Adaptation

Abstract

We propose a novel unsupervised domain adaptation framework based on domain-specific batch normalization in deep neural networks. We aim to adapt to both domains by specializing batch normalization layers in convolutional neural networks while allowing them to share all other model parameters, which is realized by a two-stage algorithm. In the first stage, we estimate pseudo-labels for the examples in the target domain using an external unsupervised domain adaptation algorithm---for example, MSTN or CPUA---integrating the proposed domain-specific batch normalization. The second stage learns the final models using a multi-task classification loss for the source and target domains. Note that the two domains have separate batch normalization layers in both stages. Our framework can be easily incorporated into the domain adaptation techniques based on deep neural networks with batch normalization layers. We also present that our approach can be extended to the problem with multiple source domains. The proposed algorithm is evaluated on multiple benchmark datasets and achieves the state-of-the-art accuracy in the standard setting and the multi-source domain adaption scenario.

References in corpus (10)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Conditional Adversarial Domain Adaptation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Domain Separation Networks
• VisDA: The Visual Domain Adaptation Challenge
• Asymmetric Tri-training for Unsupervised Domain Adaptation
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• Adversarial Alignment of Class Prediction Uncertainties for Domain Adaptation

Cited by in corpus (5)

• Rethinking "Batch" in BatchNorm
• Towards Discriminative Representation Learning for Unsupervised Person Re-identification
• Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absent
• Domain Conditioned Adaptation Network
• Consistent Batch Normalization for Weighted Loss in Imbalanced-Data Environment