Exploiting Both Domain-specific and Invariant Knowledge via a Win-win Transformer for Unsupervised Domain Adaptation
arXiv:2111.12941
Abstract
Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain. Most existing UDA approaches enable knowledge transfer via learning domain-invariant representation and sharing one classifier across two domains. However, ignoring the domain-specific information that are related to the task, and forcing a unified classifier to fit both domains will limit the feature expressiveness in each domain. In this paper, by observing that the Transformer architecture with comparable parameters can generate more transferable representations than CNN counterparts, we propose a Win-Win TRansformer framework (WinTR) that separately explores the domain-specific knowledge for each domain and meanwhile interchanges cross-domain knowledge. Specifically, we learn two different mappings using two individual classification tokens in the Transformer, and design for each one a domain-specific classifier. The cross-domain knowledge is transferred via source guided label refinement and single-sided feature alignment with respect to source or target, which keeps the integrity of domain-specific information. Extensive experiments on three benchmark datasets show that our method outperforms the state-of-the-art UDA methods, validating the effectiveness of exploiting both domain-specific and invariant
References in corpus (10)
- Bootstrap your own latent: A new approach to self-supervised Learning
- Learning Transferable Features with Deep Adaptation Networks
- VisDA: The Visual Domain Adaptation Challenge
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
- Deep Residual Correction Network for Partial Domain Adaptation
- CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
- Generalized Domain Conditioned Adaptation Network
- CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
- VOLO: Vision Outlooker for Visual Recognition
- ConvNets vs. Transformers: Whose Visual Representations are More Transferable?