Context-Aware Mixup for Domain Adaptive Semantic Segmentation

Abstract

Unsupervised domain adaptation (UDA) aims to adapt a model of the labeled source domain to an unlabeled target domain. Existing UDA-based semantic segmentation approaches always reduce the domain shifts in pixel level, feature level, and output level. However, almost all of them largely neglect the contextual dependency, which is generally shared across different domains, leading to less-desired performance. In this paper, we propose a novel Context-Aware Mixup (CAMix) framework for domain adaptive semantic segmentation, which exploits this important clue of context-dependency as explicit prior knowledge in a fully end-to-end trainable manner for enhancing the adaptability toward the target domain. Firstly, we present a contextual mask generation strategy by leveraging the accumulated spatial distributions and prior contextual relationships. The generated contextual mask is critical in this work and will guide the context-aware domain mixup on three different levels. Besides, provided the context knowledge, we introduce a significance-reweighted consistency loss to penalize the inconsistency between the mixed student prediction and the mixed teacher prediction, which alleviates the negative transfer of the adaptation, e.g., early performance degradation. Extensive experiments and analysis demonstrate the effectiveness of our method against the state-of-the-art approaches on widely-used UDA benchmarks.

Accepted to IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

References in corpus (6)

• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• Generative Domain Adaptation for Face Anti-Spoofing
• Affinity Space Adaptation for Semantic Segmentation Across Domains
• PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation
• Domain Adaptive Semantic Segmentation via Regional Contrastive Consistency Regularization

Cited by in corpus (10)

• Instance-Aware Domain Generalization for Face Anti-Spoofing
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• Generative Domain Adaptation for Face Anti-Spoofing
• PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation
• Rethinking Domain Generalization: Discriminability and Generalizability
• Crowd Localization from Gaussian Mixture Scoped Knowledge and Scoped Teacher
• Accurate identification and measurement of the precipitate area by two-stage deep neural networks in novel chromium-based alloys
• DGMamba: Domain Generalization via Generalized State Space Model
• ECAP: Extensive Cut-and-Paste Augmentation for Unsupervised Domain Adaptive Semantic Segmentation
• BCMDA: Bidirectional Correlation Maps Domain Adaptation for Mixed Domain Semi-Supervised Medical Image Segmentation