Context-Aware Domain Adaptation in Semantic Segmentation

Abstract

In this paper, we consider the problem of unsupervised domain adaptation in the semantic segmentation. There are two primary issues in this field, i.e., what and how to transfer domain knowledge across two domains. Existing methods mainly focus on adapting domain-invariant features (what to transfer) through adversarial learning (how to transfer). Context dependency is essential for semantic segmentation, however, its transferability is still not well understood. Furthermore, how to transfer contextual information across two domains remains unexplored. Motivated by this, we propose a cross-attention mechanism based on self-attention to capture context dependencies between two domains and adapt transferable context. To achieve this goal, we design two cross-domain attention modules to adapt context dependencies from both spatial and channel views. Specifically, the spatial attention module captures local feature dependencies between each position in the source and target image. The channel attention module models semantic dependencies between each pair of cross-domain channel maps. To adapt context dependencies, we further selectively aggregate the context information from two domains. The superiority of our method over existing state-of-the-art methods is empirically proved on "GTA5 to Cityscapes" and "SYNTHIA to Cityscapes".

10 pages, 6 figures, 5 tables

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Learning Transferable Features with Deep Adaptation Networks
• Self-Attention Generative Adversarial Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• A Structured Self-attentive Sentence Embedding
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach
• Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation
• Significance-aware Information Bottleneck for Domain Adaptive Semantic Segmentation

Cited by in corpus (3)

• Source-Free Domain Adaptation for Semantic Segmentation
• Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
• Exploiting Inter-pixel Correlations in Unsupervised Domain Adaptation for Semantic Segmentation