Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation

Abstract

Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. Present UDA models focus on alleviating the domain shift by minimizing the feature discrepancy between the source domain and the target domain but usually ignore the class confusion problem. In this work, we propose an Inter-class Separation and Intra-class Aggregation (ISIA) mechanism. It encourages the cross-domain representative consistency between the same categories and differentiation among diverse categories. In this way, the features belonging to the same categories are aligned together and the confusable categories are separated. By measuring the align complexity of each category, we design an Adaptive-weighted Instance Matching (AIM) strategy to further optimize the instance-level adaptation. Based on our proposed methods, we also raise a hierarchical unsupervised domain adaptation framework for cross-domain semantic segmentation task. Through performing the image-level, feature-level, category-level and instance-level alignment, our method achieves a stronger generalization performance of the model from the source domain to the target domain. In two typical cross-domain semantic segmentation tasks, i.e., GTA5 to Cityscapes and SYNTHIA to Cityscapes, our method achieves the state-of-the-art segmentation accuracy. We also build two cross-domain semantic segmentation datasets based on the publicly available data, i.e., remote sensing building segmentation and road segmentation, for domain adaptive segmentation.

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
• Fully Convolutional Networks for Semantic Segmentation
• Learning Transferable Features with Deep Adaptation Networks
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• OCNet: Object Context Network for Scene Parsing
• Unsupervised Domain Adaptation using Generative Adversarial Networks for Semantic Segmentation of Aerial Images
• Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing
• DAugNet: Unsupervised, Multi-source, Multi-target, and Life-long Domain Adaptation for Semantic Segmentation of Satellite Images
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• Affinity Space Adaptation for Semantic Segmentation Across Domains
• Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds
• Causal Generative Domain Adaptation Networks
• SceneAdapt: Scene-based domain adaptation for semantic segmentation using adversarial learning
• LiDARNet: A Boundary-Aware Domain Adaptation Model for Point Cloud Semantic Segmentation

Cited by in corpus (4)

• A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application
• A large-scale climate-aware satellite image dataset for domain adaptive land-cover semantic segmentation
• Learning at a Glance: Towards Interpretable Data-limited Continual Semantic Segmentation via Semantic-Invariance Modelling
• Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Images