Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation

Abstract

Self-training is a competitive approach in domain adaptive segmentation, which trains the network with the pseudo labels on the target domain. However inevitably, the pseudo labels are noisy and the target features are dispersed due to the discrepancy between source and target domains. In this paper, we rely on representative prototypes, the feature centroids of classes, to address the two issues for unsupervised domain adaptation. In particular, we take one step further and exploit the feature distances from prototypes that provide richer information than mere prototypes. Specifically, we use it to estimate the likelihood of pseudo labels to facilitate online correction in the course of training. Meanwhile, we align the prototypical assignments based on relative feature distances for two different views of the same target, producing a more compact target feature space. Moreover, we find that distilling the already learned knowledge to a self-supervised pretrained model further boosts the performance. Our method shows tremendous performance advantage over state-of-the-art methods. We will make the code publicly available.

References in corpus (21)

• Distilling the Knowledge in a Neural Network
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• A Simple Framework for Contrastive Learning of Visual Representations
• Prototypical Networks for Few-shot Learning
• Bootstrap your own latent: A new approach to self-supervised Learning
• Learning Transferable Features with Deep Adaptation Networks
• Improved Regularization of Convolutional Neural Networks with Cutout
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Unsupervised Domain Adaptation through Self-Supervision
• Self-supervised Domain Adaptation for Computer Vision Tasks
• Universal Domain Adaptation through Self Supervision
• On Learning Invariant Representation for Domain Adaptation
• Co-regularized Alignment for Unsupervised Domain Adaptation
• Unsupervised Scene Adaptation with Memory Regularization in vivo
• Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
• Self-labelling via simultaneous clustering and representation learning
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation
• Keep it Simple: Image Statistics Matching for Domain Adaptation
• ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks

Cited by in corpus (9)

• Robust Mutual Learning for Semi-supervised Semantic Segmentation
• Exploiting Negative Learning for Implicit Pseudo Label Rectification in Source-Free Domain Adaptive Semantic Segmentation
• Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
• Multiple Fusion Adaptation: A Strong Framework for Unsupervised Semantic Segmentation Adaptation
• Semi-supervised Domain Adaptation for Semantic Segmentation
• Exploiting Inter-pixel Correlations in Unsupervised Domain Adaptation for Semantic Segmentation
• Leveraging Domain Adaptation for Low-Resource Geospatial Machine Learning
• ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation
• MetaPix: Domain Transfer for Semantic Segmentation by Meta Pixel Weighting