Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation

Abstract

Unsupervised domain adaptation (UDA) aims to adapt existing models of the source domain to a new target domain with only unlabeled data. Most existing methods suffer from noticeable negative transfer resulting from either the error-prone discriminator network or the unreasonable teacher model. Besides, the local regional consistency in UDA has been largely neglected, and only extracting the global-level pattern information is not powerful enough for feature alignment due to the abuse use of contexts. To this end, we propose an uncertainty-aware consistency regularization method for cross-domain semantic segmentation. Firstly, we introduce an uncertainty-guided consistency loss with a dynamic weighting scheme by exploiting the latent uncertainty information of the target samples. As such, more meaningful and reliable knowledge from the teacher model can be transferred to the student model. We further reveal the reason why the current consistency regularization is often unstable in minimizing the domain discrepancy. Besides, we design a ClassDrop mask generation algorithm to produce strong class-wise perturbations. Guided by this mask, we propose a ClassOut strategy to realize effective regional consistency in a fine-grained manner. Experiments demonstrate that our method outperforms the state-of-the-art methods on four domain adaptation benchmarks, i.e., GTAV $\to$ Cityscapes and SYNTHIA $\to$ Cityscapes, Virtual KITTI $\to$ KITTI and Cityscapes $\to$ KITTI.

References in corpus (25)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation by Backpropagation
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Virtual Worlds as Proxy for Multi-Object Tracking Analysis
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• DMT: Dynamic Mutual Training for Semi-Supervised Learning
• Semi-supervised semantic segmentation needs strong, varied perturbations
• Context-Aware Mixup for Domain Adaptive Semantic Segmentation
• Expectation-Maximization Attention Networks for Semantic Segmentation
• Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
• Multi-source Domain Adaptation for Semantic Segmentation
• Affinity Space Adaptation for Semantic Segmentation Across Domains
• Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach
• PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation
• SPIGAN: Privileged Adversarial Learning from Simulation
• Unsupervised Domain Adaptation via Calibrating Uncertainties
• Self-Adversarial Disentangling for Specific Domain Adaptation
• Bayesian Uncertainty Matching for Unsupervised Domain Adaptation

Cited by in corpus (11)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Context-Aware Mixup for Domain Adaptive Semantic Segmentation
• Instance-Aware Domain Generalization for Face Anti-Spoofing
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
• Generative Domain Adaptation for Face Anti-Spoofing
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• Rethinking Domain Generalization: Discriminability and Generalizability
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• Anatomy-guided domain adaptation for 3D in-bed human pose estimation
• Self-Adversarial Disentangling for Specific Domain Adaptation
• ECAP: Extensive Cut-and-Paste Augmentation for Unsupervised Domain Adaptive Semantic Segmentation