FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation

Abstract

Fully convolutional models for dense prediction have proven successful for a wide range of visual tasks. Such models perform well in a supervised setting, but performance can be surprisingly poor under domain shifts that appear mild to a human observer. For example, training on one city and testing on another in a different geographic region and/or weather condition may result in significantly degraded performance due to pixel-level distribution shift. In this paper, we introduce the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems. Our method consists of both global and category specific adaptation techniques. Global domain alignment is performed using a novel semantic segmentation network with fully convolutional domain adversarial learning. This initially adapted space then enables category specific adaptation through a generalization of constrained weak learning, with explicit transfer of the spatial layout from the source to the target domains. Our approach outperforms baselines across different settings on multiple large-scale datasets, including adapting across various real city environments, different synthetic sub-domains, from simulated to real environments, and on a novel large-scale dash-cam dataset.

References in corpus (5)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Unsupervised Domain Adaptation by Backpropagation
• Learning Deconvolution Network for Semantic Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• Playing for Data: Ground Truth from Computer Games

Cited by in corpus (84)

• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• VisDA: The Visual Domain Adaptation Challenge
• Patch-based Output Space Adversarial Learning for Joint Optic Disc and Cup Segmentation
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
• Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes
• On Learning Invariant Representation for Domain Adaptation
• Contrastive Adaptation Network for Unsupervised Domain Adaptation
• Unsupervised domain adaptation semantic segmentation of high-resolution remote sensing imagery with invariant domain-level prototype memory
• Complementary Pseudo Labels For Unsupervised Domain Adaptation On Person Re-identification
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• How much real data do we actually need: Analyzing object detection performance using synthetic and real data
• Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation
• FDA: Fourier Domain Adaptation for Semantic Segmentation
• Label Efficient Learning of Transferable Representations across Domains and Tasks
• One-Sided Unsupervised Domain Mapping
• Domain Adaptation for Semantic Segmentation with Maximum Squares Loss
• I2F: A Unified Image-to-Feature Approach for Domain Adaptive Semantic Segmentation
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• FSDR: Frequency Space Domain Randomization for Domain Generalization
• Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learning
• Image to Image Translation for Domain Adaptation
• Significance-aware Information Bottleneck for Domain Adaptive Semantic Segmentation
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation
• What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation
• SceneAdapt: Scene-based domain adaptation for semantic segmentation using adversarial learning
• An Adversarial Regularisation for Semi-Supervised Training of Structured Output Neural Networks
• DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception
• Keep it Simple: Image Statistics Matching for Domain Adaptation
• Condition-Invariant Semantic Segmentation
• Multi-Target Domain Adaptation with Collaborative Consistency Learning
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Cross-View Regularization for Domain Adaptive Panoptic Segmentation
• Style Transfer with Diffusion Models for Synthetic-to-Real Domain Adaptation
• Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection
• Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation
• VisDA-2021 Competition Universal Domain Adaptation to Improve Performance on Out-of-Distribution Data
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• Unsupervised Domain Adaptation for Semantic Segmentation via Low-level Edge Information Transfer
• Convex Shape Prior for Deep Neural Convolution Network based Eye Fundus Images Segmentation
• DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation
• An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation
• Exploiting Negative Learning for Implicit Pseudo Label Rectification in Source-Free Domain Adaptive Semantic Segmentation
• Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation
• Weakly-supervised Caricature Face Parsing through Domain Adaptation
• DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
• Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment
• Generalizing Deep Models for Overhead Image Segmentation Through Getis-Ord Gi* Pooling
• Phase Consistent Ecological Domain Adaptation
• CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation
• MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation
• AutoAdapt: Automated Segmentation Network Search for Unsupervised Domain Adaptation
• Towards Adaptive Semantic Segmentation by Progressive Feature Refinement
• Coarse-to-Fine Domain Adaptive Semantic Segmentation with Photometric Alignment and Category-Center Regularization
• Light-weight Calibrator: a Separable Component for Unsupervised Domain Adaptation
• Multi-Anchor Active Domain Adaptation for Semantic Segmentation
• Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation
• Learning to simulate complex scenes
• Unsupervised Classification of Street Architectures Based on InfoGAN
• Learn distributed GAN with Temporary Discriminators
• Two-phase Pseudo Label Densification for Self-training based Domain Adaptation
• Transferring and Regularizing Prediction for Semantic Segmentation
• Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
• Restyling Data: Application to Unsupervised Domain Adaptation
• LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentation
• Semi-supervised Domain Adaptation for Semantic Segmentation
• Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources
• Domain Adaptation on Semantic Segmentation for Aerial Images
• Scale Aware Adaptation for Land-Cover Classification in Remote Sensing Imagery
• What Synthesis is Missing: Depth Adaptation Integrated with Weak Supervision for Indoor Scene Parsing
• One-shot Unsupervised Domain Adaptation with Personalized Diffusion Models
• Domain Adaptation on Semantic Segmentation with Separate Affine Transformation in Batch Normalization
• Multichannel Semantic Segmentation with Unsupervised Domain Adaptation
• Generative AI for Enhanced Wildfire Detection: Bridging the Synthetic-Real Domain Gap
• Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrast
• Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency
• From Real to Synthetic and Back: Synthesizing Training Data for Multi-Person Scene Understanding
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation
• Synthetic Convolutional Features for Improved Semantic Segmentation
• Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentation
• CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency
• On-target Adaptation