Spectral Unsupervised Domain Adaptation for Visual Recognition
arXiv:2106.06112
Abstract
Though unsupervised domain adaptation (UDA) has achieved very impressive progress recently, it remains a great challenge due to missing target annotations and the rich discrepancy between source and target distributions. We propose Spectral UDA (SUDA), an effective and efficient UDA technique that works in the spectral space and can generalize across different visual recognition tasks. SUDA addresses the UDA challenges from two perspectives. First, it introduces a spectrum transformer (ST) that mitigates inter-domain discrepancies by enhancing domain-invariant spectra while suppressing domain-variant spectra of source and target samples simultaneously. Second, it introduces multi-view spectral learning that learns useful unsupervised representations by maximizing mutual information among multiple ST-generated spectral views of each target sample. Extensive experiments show that SUDA achieves superior accuracy consistently across different visual tasks in object detection, semantic segmentation and image classification. Additionally, SUDA also works with the transformer-based network and achieves state-of-the-art performance on object detection.
arXiv commentAccepted to CVPR2022
References in corpus (15)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Learning Transferable Features with Deep Adaptation Networks
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
- Semantic Foggy Scene Understanding with Synthetic Data
- Unsupervised Domain Adaptation with Residual Transfer Networks
- Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks
- Fourier Spectrum Discrepancies in Deep Network Generated Images
- Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision
- Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning
- Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation
- FSDR: Frequency Space Domain Randomization for Domain Generalization
- MLAN: Multi-Level Adversarial Network for Domain Adaptive Semantic Segmentation
- Cross-View Regularization for Domain Adaptive Panoptic Segmentation