CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation

Abstract

Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to a different unlabeled target domain. Most existing UDA methods focus on learning domain-invariant feature representation, either from the domain level or category level, using convolution neural networks (CNNs)-based frameworks. One fundamental problem for the category level based UDA is the production of pseudo labels for samples in target domain, which are usually too noisy for accurate domain alignment, inevitably compromising the UDA performance. With the success of Transformer in various tasks, we find that the cross-attention in Transformer is robust to the noisy input pairs for better feature alignment, thus in this paper Transformer is adopted for the challenging UDA task. Specifically, to generate accurate input pairs, we design a two-way center-aware labeling algorithm to produce pseudo labels for target samples. Along with the pseudo labels, a weight-sharing triple-branch transformer framework is proposed to apply self-attention and cross-attention for source/target feature learning and source-target domain alignment, respectively. Such design explicitly enforces the framework to learn discriminative domain-specific and domain-invariant representations simultaneously. The proposed method is dubbed CDTrans (cross-domain transformer), and it provides one of the first attempts to solve UDA tasks with a pure transformer solution. Experiments show that our proposed method achieves the best performance on public UDA datasets, e.g. VisDA-2017 and DomainNet. Code and models are available at https://github.com/CDTrans/CDTrans.

References in corpus (21)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Distilling the Knowledge in a Neural Network
• A Survey on Visual Transformer
• Transformers in Vision: A Survey
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Transformer in Transformer
• VisDA: The Visual Domain Adaptation Challenge
• DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
• Intriguing Properties of Vision Transformers
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• XCiT: Cross-Covariance Image Transformers
• Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
• Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
• TransReID: Transformer-based Object Re-Identification
• Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation
• Refiner: Refining Self-attention for Vision Transformers
• VOLO: Vision Outlooker for Visual Recognition
• Transformer-Based Source-Free Domain Adaptation
• PSViT: Better Vision Transformer via Token Pooling and Attention Sharing
• KVT: k-NN Attention for Boosting Vision Transformers

Cited by in corpus (7)

• MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation
• Towards Unsupervised Domain Adaptation via Domain-Transformer
• Simulations of Common Unsupervised Domain Adaptation Algorithms for Image Classification
• Exploiting Both Domain-specific and Invariant Knowledge via a Win-win Transformer for Unsupervised Domain Adaptation
• Towards Attention-based Contrastive Learning for Audio Spoof Detection
• BiPC: Bidirectional Probability Calibration for Unsupervised Domain Adaption
• A Low-Cost Neural ODE with Depthwise Separable Convolution for Edge Domain Adaptation on FPGAs