No More Discrimination: Cross City Adaptation of Road Scene Segmenters

Abstract

Despite the recent success of deep-learning based semantic segmentation, deploying a pre-trained road scene segmenter to a city whose images are not presented in the training set would not achieve satisfactory performance due to dataset biases. Instead of collecting a large number of annotated images of each city of interest to train or refine the segmenter, we propose an unsupervised learning approach to adapt road scene segmenters across different cities. By utilizing Google Street View and its time-machine feature, we can collect unannotated images for each road scene at different times, so that the associated static-object priors can be extracted accordingly. By advancing a joint global and class-specific domain adversarial learning framework, adaptation of pre-trained segmenters to that city can be achieved without the need of any user annotation or interaction. We show that our method improves the performance of semantic segmentation in multiple cities across continents, while it performs favorably against state-of-the-art approaches requiring annotated training data.

arXiv comment13 pages, 10 figures

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation by Backpropagation
• Multi-Scale Context Aggregation by Dilated Convolutions
• Unsupervised Domain Adaptation with Residual Transfer Networks
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Fully Convolutional Networks for Semantic Segmentation
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Learning Deconvolution Network for Semantic Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
• Playing for Data: Ground Truth from Computer Games
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer