Effective Use of Synthetic Data for Urban Scene Semantic Segmentation

Abstract

Training a deep network to perform semantic segmentation requires large amounts of labeled data. To alleviate the manual effort of annotating real images, researchers have investigated the use of synthetic data, which can be labeled automatically. Unfortunately, a network trained on synthetic data performs relatively poorly on real images. While this can be addressed by domain adaptation, existing methods all require having access to real images during training. In this paper, we introduce a drastically different way to handle synthetic images that does not require seeing any real images at training time. Our approach builds on the observation that foreground and background classes are not affected in the same manner by the domain shift, and thus should be treated differently. In particular, the former should be handled in a detection-based manner to better account for the fact that, while their texture in synthetic images is not photo-realistic, their shape looks natural. Our experiments evidence the effectiveness of our approach on Cityscapes and CamVid with models trained on synthetic data only.

Accepted in European Conference on Computer Vision (ECCV), 2018

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Learning Deconvolution Network for Semantic Segmentation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• CARLA: An Open Urban Driving Simulator
• Fully Convolutional Multi-Class Multiple Instance Learning
• Playing for Data: Ground Truth from Computer Games
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
• Exploiting saliency for object segmentation from image level labels
• Image to Image Translation for Domain Adaptation

Cited by in corpus (6)

• All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentation
• ACE: Adapting to Changing Environments for Semantic Segmentation
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• Transferring and Regularizing Prediction for Semantic Segmentation
• Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation
• Manual-Label Free 3D Detection via An Open-Source Simulator