Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes

Abstract

Semantic segmentation, a pixel-level vision task, is developed rapidly by using convolutional neural networks (CNNs). Training CNNs requires a large amount of labeled data, but manually annotating data is difficult. For emancipating manpower, in recent years, some synthetic datasets are released. However, they are still different from real scenes, which causes that training a model on the synthetic data (source domain) cannot achieve a good performance on real urban scenes (target domain). In this paper, we propose a weakly supervised adversarial domain adaptation to improve the segmentation performance from synthetic data to real scenes, which consists of three deep neural networks. To be specific, a detection and segmentation ("DS" for short) model focuses on detecting objects and predicting segmentation map; a pixel-level domain classifier ("PDC" for short) tries to distinguish the image features from which domains; an object-level domain classifier ("ODC" for short) discriminates the objects from which domains and predicts the objects classes. PDC and ODC are treated as the discriminators, and DS is considered as the generator. By adversarial learning, DS is supposed to learn domain-invariant features. In experiments, our proposed method yields the new record of mIoU metric in the same problem.

To appear at TIP

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Generative Adversarial Networks
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation by Backpropagation
• Conditional Random Fields as Recurrent Neural Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Multi-Scale Context Aggregation by Dilated Convolutions
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Domain Separation Networks
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Embedding Structured Contour and Location Prior in Siamesed Fully Convolutional Networks for Road Detection

Cited by in corpus (16)

• JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation
• SCAR: Spatial-/Channel-wise Attention Regression Networks for Crowd Counting
• Localizing Anomalies from Weakly-Labeled Videos
• Road Segmentation for Remote Sensing Images using Adversarial Spatial Pyramid Networks
• Few-shot Object Detection on Remote Sensing Images
• Night-time Scene Parsing with a Large Real Dataset
• Gaussian Dynamic Convolution for Efficient Single-Image Segmentation
• Affinity Space Adaptation for Semantic Segmentation Across Domains
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• Learning causal representations for robust domain adaptation
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• Feature-aware Adaptation and Density Alignment for Crowd Counting in Video Surveillance
• Information Bottleneck Theory on Convolutional Neural Networks
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation
• Importance Filtered Cross-Domain Adaptation
• ASK: Adaptively Selecting Key Local Features for RGB-D Scene Recognition