Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation

Abstract

Training a semantic segmentation model requires a large amount of pixel-level annotation, hampering its application at scale. With computer graphics, we can generate almost unlimited training data with precise annotation. However,a deep model trained with synthetic data usually cannot directly generalize well to realistic images due to domain shift. It has been observed that highly confident labels for the unlabeled real images may be predicted relying on the labeled synthetic data. To tackle the unsupervised domain adaptation problem, we explore the possibilities to generate high-quality labels as proxy labels to supervise the training on target data. Specifically, we propose a novel proxy-based method using multi-adversarial training. We first train the model using synthetic data (source domain). Multiple discriminators are used to align the features be-tween the source and target domain (real images) at different levels. Then we focus on obtaining and selecting high-quality proxy labels by incorporating both the confidence of the class predictor and that from the adversarial discriminators. Our discriminators not only work as a regularizer to encourage feature alignment but also provide an alternative confidence measure for generating proxy labels. Relying on the generated high-quality proxies, our model can be trained in a "supervised manner" on the target do-main. On two major tasks, GTA5->Cityscapes and SYNTHIA->Cityscapes, our method achieves state-of-the-art results, outperforming the previous by a large margin.

References in corpus (13)

• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Unsupervised Domain Adaptation with Residual Transfer Networks
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• Fully Convolutional Adaptation Networks for Semantic Segmentation
• Weakly Supervised Semantic Segmentation using Web-Crawled Videos
• Adaptive Semantic Segmentation with a Strategic Curriculum of Proxy Labels

Cited by in corpus (4)

• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• Learning to Zoom-in via Learning to Zoom-out: Real-world Super-resolution by Generating and Adapting Degradation
• One-shot Unsupervised Domain Adaptation with Personalized Diffusion Models
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation