Affinity Space Adaptation for Semantic Segmentation Across Domains

Abstract

Semantic segmentation with dense pixel-wise annotation has achieved excellent performance thanks to deep learning. However, the generalization of semantic segmentation in the wild remains challenging. In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation. Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains by leveraging co-occurring patterns between pairwise pixels in the output of structured semantic segmentation. This is different from most existing approaches that attempt to adapt domains based on individual pixel-wise information in image, feature, or output level. Specifically, we perform domain adaptation on the affinity relationship between adjacent pixels termed affinity space of source and target domain. To this end, we develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment. Extensive experiments demonstrate that the proposed method achieves superior performance against some state-of-the-art methods on several challenging benchmarks for semantic segmentation across domains. The code is available at https://github.com/idealwei/ASANet.

Accepted by IEEE TIP

References in corpus (5)

• Conditional Random Fields as Recurrent Neural Networks
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Semantic Foggy Scene Understanding with Synthetic Data
• REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs
• Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes

Cited by in corpus (9)

• Context-Aware Mixup for Domain Adaptive Semantic Segmentation
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• Unsupervised Foggy Scene Understanding via Self Spatial-Temporal Label Diffusion
• Domain Adaptive Semantic Segmentation via Regional Contrastive Consistency Regularization
• Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation
• A large-scale climate-aware satellite image dataset for domain adaptive land-cover semantic segmentation
• Single-Image Super-Resolution Reconstruction based on the Differences of Neighboring Pixels