Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Abstract

Domain adaptive semantic segmentation aims to train a model performing satisfactory pixel-level predictions on the target with only out-of-domain (source) annotations. The conventional solution to this task is to minimize the discrepancy between source and target to enable effective knowledge transfer. Previous domain discrepancy minimization methods are mainly based on the adversarial training. They tend to consider the domain discrepancy globally, which ignore the pixel-wise relationships and are less discriminative. In this paper, we propose to build the pixel-level cycle association between source and target pixel pairs and contrastively strengthen their connections to diminish the domain gap and make the features more discriminative. To the best of our knowledge, this is a new perspective for tackling such a challenging task. Experiment results on two representative domain adaptation benchmarks, i.e. GTAV $\to$ Cityscapes and SYNTHIA $\to$ Cityscapes, verify the effectiveness of our proposed method and demonstrate that our method performs favorably against previous state-of-the-arts. Our method can be trained end-to-end in one stage and introduces no additional parameters, which is expected to serve as a general framework and help ease future research in domain adaptive semantic segmentation. Code is available at https://github.com/kgl-prml/Pixel- Level-Cycle-Association.

Accepted by NeurIPS 2020 (oral). Code: https://github.com/kgl-prml/Pixel- Level-Cycle-Association

References in corpus (7)

• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Unsupervised Scene Adaptation with Memory Regularization in vivo
• Contrastive Representation Distillation
• Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach
• Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation

Cited by in corpus (14)

• A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images
• Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation
• Relation Matters: Foreground-aware Graph-based Relational Reasoning for Domain Adaptive Object Detection
• Few-Shot Segmentation via Cycle-Consistent Transformer
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation
• Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation
• Semantic Distribution-aware Contrastive Adaptation for Semantic Segmentation
• ReCLIP++: Learn to Rectify the Bias of CLIP for Unsupervised Semantic Segmentation
• SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning
• WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation
• Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation
• Unsupervised Domain Adaptation in Semantic Segmentation Based on Pixel Alignment and Self-Training
• Exploiting Inter-pixel Correlations in Unsupervised Domain Adaptation for Semantic Segmentation
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation