Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation

Abstract

Domain adaptation for semantic segmentation aims to improve the model performance in the presence of a distribution shift between source and target domain. Leveraging the supervision from auxiliary tasks~(such as depth estimation) has the potential to heal this shift because many visual tasks are closely related to each other. However, such a supervision is not always available. In this work, we leverage the guidance from self-supervised depth estimation, which is available on both domains, to bridge the domain gap. On the one hand, we propose to explicitly learn the task feature correlation to strengthen the target semantic predictions with the help of target depth estimation. On the other hand, we use the depth prediction discrepancy from source and target depth decoders to approximate the pixel-wise adaptation difficulty. The adaptation difficulty, inferred from depth, is then used to refine the target semantic segmentation pseudo-labels. The proposed method can be easily implemented into existing segmentation frameworks. We demonstrate the effectiveness of our approach on the benchmark tasks SYNTHIA-to-Cityscapes and GTA-to-Cityscapes, on which we achieve the new state-of-the-art performance of $55.0\%$ and $56.6\%$, respectively. Our code is available at \url{https://qin.ee/corda}.

arXiv commentTo appear in ICCV 2021

References in corpus (12)

• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Multi-Task Learning for Dense Prediction Tasks: A Survey
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
• Unsupervised Domain Adaptation through Self-Supervision
• Self-supervised Domain Adaptation for Computer Vision Tasks
• Unsupervised Scene Adaptation with Memory Regularization in vivo
• Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
• Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation
• Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation
• Self-Supervised Relative Depth Learning for Urban Scene Understanding
• Context-Aware Domain Adaptation in Semantic Segmentation