Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment

Abstract

The supervised training of deep networks for semantic segmentation requires a huge amount of labeled real world data. To solve this issue, a commonly exploited workaround is to use synthetic data for training, but deep networks show a critical performance drop when analyzing data with slightly different statistical properties with respect to the training set. In this work, we propose a novel Unsupervised Domain Adaptation (UDA) strategy to address the domain shift issue between real world and synthetic representations. An adversarial model, based on the cycle consistency framework, performs the mapping between the synthetic and real domain. The data is then fed to a MobileNet-v2 architecture that performs the semantic segmentation task. An additional couple of discriminators, working at the feature level of the MobileNet-v2, allows to better align the features of the two domain distributions and to further improve the performance. Finally, the consistency of the semantic maps is exploited. After an initial supervised training on synthetic data, the whole UDA architecture is trained end-to-end considering all its components at once. Experimental results show how the proposed strategy is able to obtain impressive performance in adapting a segmentation network trained on synthetic data to real world scenarios. The usage of the lightweight MobileNet-v2 architecture allows its deployment on devices with limited computational resources as the ones employed in autonomous vehicles.

arXiv comment11 pages, 3 figures, 3 tables

References in corpus (10)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Unsupervised Domain Adaptation with Residual Transfer Networks
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation
• Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach
• ACE: Adapting to Changing Environments for Semantic Segmentation

Cited by in corpus (9)

• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• Knowledge Distillation Methods for Efficient Unsupervised Adaptation Across Multiple Domains
• Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation
• Generative AI for Enhanced Wildfire Detection: Bridging the Synthetic-Real Domain Gap
• One-shot Unsupervised Domain Adaptation with Personalized Diffusion Models
• TridentAdapt: Learning Domain-invariance via Source-Target Confrontation and Self-induced Cross-domain Augmentation
• Carton dataset synthesis method for domain shift based on foreground texture decoupling and replacement
• DA-Cal: Towards Cross-Domain Calibration in Semantic Segmentation