Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy

Abstract

Reducing feature redundancy has shown beneficial effects for improving the accuracy of deep learning models, thus it is also indispensable for the models of unsupervised domain adaptation (UDA). Nevertheless, most recent efforts in the field of UDA ignores this point. Moreover, main schemes realizing this in general independent of UDA purely involve a single domain, thus might not be effective for cross-domain tasks. In this paper, we emphasize the significance of reducing feature redundancy for improving UDA in a bi-level way. For the first level, we try to ensure compact domain-specific features with a transferable decorrelated normalization module, which preserves specific domain information whilst easing the side effect of feature redundancy on the sequel domain-invariance. In the second level, domain-invariant feature redundancy caused by domain-shared representation is further mitigated via an alternative brand orthogonality for better generalization. These two novel aspects can be easily plugged into any BN-based backbone neural networks. Specifically, simply applying them to ResNet50 has achieved competitive performance to the state-of-the-arts on five popular benchmarks. Our code will be available at https://github.com/dreamkily/gUDA.

arXiv comment12 pages, 5 figures

References in corpus (15)

• Deep Domain Confusion: Maximizing for Domain Invariance
• Conditional Adversarial Domain Adaptation
• VisDA: The Visual Domain Adaptation Challenge
• Adversarial Discriminative Domain Adaptation
• Maximum Density Divergence for Domain Adaptation
• Wasserstein Distance Guided Representation Learning for Domain Adaptation
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• Multi-Adversarial Domain Adaptation
• Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation
• A Mean Field Theory of Batch Normalization
• Decorrelated Batch Normalization
• All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation
• Transfer Learning with Label Noise
• Self-adaptive Re-weighted Adversarial Domain Adaptation
• Controllable Orthogonalization in Training DNNs