Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation

Abstract

Unsupervised domain adaptation (DA) has gained substantial interest in semantic segmentation. However, almost all prior arts assume concurrent access to both labeled source and unlabeled target, making them unsuitable for scenarios demanding source-free adaptation. In this work, we enable source-free DA by partitioning the task into two: a) source-only domain generalization and b) source-free target adaptation. Towards the former, we provide theoretical insights to develop a multi-head framework trained with a virtually extended multi-source dataset, aiming to balance generalization and specificity. Towards the latter, we utilize the multi-head framework to extract reliable target pseudo-labels for self-training. Additionally, we introduce a novel conditional prior-enforcing auto-encoder that discourages spatial irregularities, thereby enhancing the pseudo-label quality. Experiments on the standard GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes benchmarks show our superiority even against the non-source-free prior-arts. Further, we show our compatibility with online adaptation enabling deployment in a sequentially changing environment.

arXiv commentICCV 2021. Project page: http://sites.google.com/view/sfdaseg

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Model Adaptation: Unsupervised Domain Adaptation without Source Data
• Fast Patch-based Style Transfer of Arbitrary Style
• Domain Agnostic Learning with Disentangled Representations
• Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
• Style Augmentation: Data Augmentation via Style Randomization
• Unsupervised Scene Adaptation with Memory Regularization in vivo
• Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
• Multi-source Domain Adaptation for Semantic Segmentation
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation
• Source-Relaxed Domain Adaptation for Image Segmentation
• Your Classifier can Secretly Suffice Multi-Source Domain Adaptation
• Automated Synthetic-to-Real Generalization
• Semantically Adaptive Image-to-image Translation for Domain Adaptation of Semantic Segmentation
• Unsupervised Domain Adaptation for Semantic Segmentation by Content Transfer

Cited by in corpus (2)

• Anatomy-guided domain adaptation for 3D in-bed human pose estimation
• Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation