Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes

Abstract

During the last half decade, convolutional neural networks (CNNs) have triumphed over semantic segmentation, which is one of the core tasks in many applications such as autonomous driving. However, to train CNNs requires a considerable amount of data, which is difficult to collect and laborious to annotate. Recent advances in computer graphics make it possible to train CNNs on photo-realistic synthetic imagery with computer-generated annotations. Despite this, the domain mismatch between the real images and the synthetic data cripples the models' performance. Hence, we propose a curriculum-style learning approach to minimize the domain gap in urban scenery semantic segmentation. The curriculum domain adaptation solves easy tasks first to infer necessary properties about the target domain; in particular, the first task is to learn global label distributions over images and local distributions over landmark superpixels. These are easy to estimate because images of urban scenes have strong idiosyncrasies (e.g., the size and spatial relations of buildings, streets, cars, etc.). We then train a segmentation network while regularizing its predictions in the target domain to follow those inferred properties. In experiments, our method outperforms the baselines on two datasets and two backbone networks. We also report extensive ablation studies about our approach.

This is the extended version of the ICCV 2017 paper "Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes" with additional GTA experiment

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• ADADELTA: An Adaptive Learning Rate Method
• Fully Convolutional Networks for Semantic Segmentation
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation by Backpropagation
• Deep Domain Confusion: Maximizing for Domain Invariance
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• Synthetic to Real Adaptation with Generative Correlation Alignment Networks
• Hierarchical Adaptive Structural SVM for Domain Adaptation

Cited by in corpus (140)

• Deep Visual Domain Adaptation: A Survey
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• The ApolloScape Open Dataset for Autonomous Driving and its Application
• Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID
• Constrained-CNN losses for weakly supervised segmentation
• Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes
• Unsupervised Domain Adaptation using Generative Adversarial Networks for Semantic Segmentation of Aerial Images
• Self-supervised Domain Adaptation for Computer Vision Tasks
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation
• Night-time Scene Parsing with a Large Real Dataset
• Context-Aware Mixup for Domain Adaptive Semantic Segmentation
• Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• Curriculum semi-supervised segmentation
• Generalized Domain Conditioned Adaptation Network
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• FDA: Fourier Domain Adaptation for Semantic Segmentation
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment
• Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training
• Adversarial Domain Adaptation with Prototype-Based Normalized Output Conditioner
• Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation
• Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision
• Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach
• Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation
• Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning
• SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud
• Diversify and Match: A Domain Adaptive Representation Learning Paradigm for Object Detection
• A Survey on Deep Learning of Small Sample in Biomedical Image Analysis
• ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks
• Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation
• Collaborative Multi-Robot Systems for Search and Rescue: Coordination and Perception
• Constrained domain adaptation for Image segmentation
• FSDR: Frequency Space Domain Randomization for Domain Generalization
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• SPIGAN: Privileged Adversarial Learning from Simulation
• Recognition in Terra Incognita
• Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
• What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation
• Transformer-Based Source-Free Domain Adaptation
• Semantic Distribution-aware Contrastive Adaptation for Semantic Segmentation
• Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data
• Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation
• Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
• Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation
• Domain Adaptation for Time Series Forecasting via Attention Sharing
• Unsupervised Real-world Image Super Resolution via Domain-distance Aware Training
• Keep it Simple: Image Statistics Matching for Domain Adaptation
• Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
• AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching
• Gradually Vanishing Bridge for Adversarial Domain Adaptation
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Train in Germany, Test in The USA: Making 3D Object Detectors Generalize
• Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer
• Domain Adaptive Detection of MAVs: A Benchmark and Noise Suppression Network
• VersatileGait: A Large-Scale Synthetic Gait Dataset with Fine-GrainedAttributes and Complicated Scenarios
• Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation
• OMNIA Faster R-CNN: Detection in the wild through dataset merging and soft distillation
• Efficient Deep Neural Networks
• Category Contrast for Unsupervised Domain Adaptation in Visual Tasks
• Semantic-Transferable Weakly-Supervised Endoscopic Lesions Segmentation
• VDM-DA: Virtual Domain Modeling for Source Data-free Domain Adaptation
• Improving Generalization of Transfer Learning Across Domains Using Spatio-Temporal Features in Autonomous Driving
• Learning Cross-domain Generalizable Features by Representation Disentanglement
• Cross-Domain Adaptation for Animal Pose Estimation
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• Mixture Domain Adaptation to Improve Semantic Segmentation in Real-World Surveillance
• Style Normalization and Restitution for Domain Generalization and Adaptation
• Domain Adaptive Medical Image Segmentation via Adversarial Learning of Disease-Specific Spatial Patterns
• DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation
• Curriculum based Dropout Discriminator for Domain Adaptation
• Distribution Regularized Self-Supervised Learning for Domain Adaptation of Semantic Segmentation
• DINE: Domain Adaptation from Single and Multiple Black-box Predictors
• Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation
• Model-Based Domain Generalization
• Adaptive Object Detection with Dual Multi-Label Prediction
• Neural networks for semantic segmentation of historical city maps: Cross-cultural performance and the impact of figurative diversity
• Uncertainty-Aware Unsupervised Domain Adaptation in Object Detection
• SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning
• Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment
• Phase Consistent Ecological Domain Adaptation
• Practical Evaluation of Out-of-Distribution Detection Methods for Image Classification
• Zero-Shot Deep Domain Adaptation
• CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation
• Generating synthetic photogrammetric data for training deep learning based 3D point cloud segmentation models
• Toddler-Guidance Learning: Impacts of Critical Period on Multimodal AI Agents
• Semantically Multi-modal Image Synthesis
• WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation
• Context-Aware Domain Adaptation in Semantic Segmentation
• MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation
• Universal Domain Adaptation in Ordinal Regression
• Hard Pixel Mining for Depth Privileged Semantic Segmentation
• ECAP: Extensive Cut-and-Paste Augmentation for Unsupervised Domain Adaptive Semantic Segmentation
• Robustified Domain Adaptation
• Towards Adaptive Semantic Segmentation by Progressive Feature Refinement
• Cross-Domain Transfer Learning with CoRTe: Consistent and Reliable Transfer from Black-Box to Lightweight Segmentation Model
• Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
• Domain-Adversarial Training of Self-Attention Based Networks for Land Cover Classification using Multi-temporal Sentinel-2 Satellite Imagery
• Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net
• Discovering Latent Classes for Semi-Supervised Semantic Segmentation
• Restyling Data: Application to Unsupervised Domain Adaptation
• Multilevel Knowledge Transfer for Cross-Domain Object Detection
• A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation
• Conditional Coupled Generative Adversarial Networks for Zero-Shot Domain Adaptation
• ICPR 2024 Competition on Domain Adaptation and GEneralization for Character Classification (DAGECC)
• Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models
• Domain-Division based Progressive Learning for Source-Free Domain Adaptation
• Consistent Posterior Distributions under Vessel-Mixing: A Regularization for Cross-Domain Retinal Artery/Vein Classification
• Unsupervised Vehicle Counting via Multiple Camera Domain Adaptation
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection
• Style Curriculum Learning for Robust Medical Image Segmentation
• Domain Adaptation on Semantic Segmentation with Separate Affine Transformation in Batch Normalization
• The NEOLIX Open Dataset for Autonomous Driving
• Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images
• Bi-Dimensional Feature Alignment for Cross-Domain Object Detection
• DRIV100: In-The-Wild Multi-Domain Dataset and Evaluation for Real-World Domain Adaptation of Semantic Segmentation
• TridentAdapt: Learning Domain-invariance via Source-Target Confrontation and Self-induced Cross-domain Augmentation
• On Direct Distribution Matching for Adapting Segmentation Networks
• A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation
• Multichannel Semantic Segmentation with Unsupervised Domain Adaptation
• What Synthesis is Missing: Depth Adaptation Integrated with Weak Supervision for Indoor Scene Parsing
• Domain-robust VQA with diverse datasets and methods but no target labels
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation
• Sparse Pose Trajectory Completion
• Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
• Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentation
• ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation
• Dynamic Adaptation on Non-Stationary Visual Domains
• Leveraging Motion Priors in Videos for Improving Human Segmentation
• CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency
• iFAN: Image-Instance Full Alignment Networks for Adaptive Object Detection
• Exploring Dropout Discriminator for Domain Adaptation
• VersatileGait: A Large-Scale Synthetic Gait Dataset Towards in-the-Wild Simulation
• Angular Gap: Reducing the Uncertainty of Image Difficulty through Model Calibration
• Unsupervised Adaptive Semantic Segmentation with Local Lipschitz Constraint
• Efficient Video Understanding via Layered Multi Frame-Rate Analysis
• Shape-conditioned Image Generation by Learning Latent Appearance Representation from Unpaired Data
• 3D Scene Parsing via Class-Wise Adaptation