Exploiting Inter-pixel Correlations in Unsupervised Domain Adaptation for Semantic Segmentation

Abstract

"Self-training" has become a dominant method for semantic segmentation via unsupervised domain adaptation (UDA). It creates a set of pseudo labels for the target domain to give explicit supervision. However, the pseudo labels are noisy, sparse and do not provide any information about inter-pixel correlations. We regard inter-pixel correlation quite important because semantic segmentation is a task of predicting highly structured pixel-level outputs. Therefore, in this paper, we propose a method of transferring the inter-pixel correlations from the source domain to the target domain via a self-attention module. The module takes the prediction of the segmentation network as an input and creates a self-attended prediction that correlates similar pixels. The module is trained only on the source domain to learn the domain-invariant inter-pixel correlations, then later, it is used to train the segmentation network on the target domain. The network learns not only from the pseudo labels but also by following the output of the self-attention module which provides additional knowledge about the inter-pixel correlations. Through extensive experiments, we show that our method significantly improves the performance on two standard UDA benchmarks and also can be combined with recent state-of-the-art method to achieve better performance.

References in corpus (19)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Deep Learning using Rectified Linear Units (ReLU)
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Fully Convolutional Networks for Semantic Segmentation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Long Short-Term Memory-Networks for Machine Reading
• Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• A Decomposable Attention Model for Natural Language Inference
• FDA: Fourier Domain Adaptation for Semantic Segmentation
• Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation
• Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation
• Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation
• Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation
• Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation
• MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation
• Context-Aware Domain Adaptation in Semantic Segmentation