Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation

Abstract

Deep convolutional neural networks have considerably improved state-of-the-art results for semantic segmentation. Nevertheless, even modern architectures lack the ability to generalize well to a test dataset that originates from a different domain. To avoid the costly annotation of training data for unseen domains, unsupervised domain adaptation (UDA) attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. Previous work has mainly focused on minimizing the discrepancy between the two domains by using adversarial training or self-training. While adversarial training may fail to align the correct semantic categories as it minimizes the discrepancy between the global distributions, self-training raises the question of how to provide reliable pseudo-labels. To align the correct semantic categories across domains, we propose a contrastive learning approach that adapts category-wise centroids across domains. Furthermore, we extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels. Although both contrastive learning and self-training (CLST) through temporal ensembling enable knowledge transfer between two domains, it is their combination that leads to a symbiotic structure. We validate our approach on two domain adaptation benchmarks: GTA5 $\to$ Cityscapes and SYNTHIA $\to$ Cityscapes. Our method achieves better or comparable results than the state-of-the-art. We will make the code publicly available.

References in corpus (16)

• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• A Simple Framework for Contrastive Learning of Visual Representations
• Bootstrap your own latent: A new approach to self-supervised Learning
• Temporal Ensembling for Semi-Supervised Learning
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• High-Resolution Representations for Labeling Pixels and Regions
• Equivalence of distance-based and RKHS-based statistics in hypothesis testing
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• Self-ensembling for visual domain adaptation
• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment
• Instance Adaptive Self-Training for Unsupervised Domain Adaptation
• Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation