FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation

Abstract

Fully convolutional models for dense prediction have proven successful for a wide range of visual tasks. Such models perform well in a supervised setting, but performance can be surprisingly poor under domain shifts that appear mild to a human observer. For example, training on one city and testing on another in a different geographic region and/or weather condition may result in significantly degraded performance due to pixel-level distribution shift. In this paper, we introduce the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems. Our method consists of both global and category specific adaptation techniques. Global domain alignment is performed using a novel semantic segmentation network with fully convolutional domain adversarial learning. This initially adapted space then enables category specific adaptation through a generalization of constrained weak learning, with explicit transfer of the spatial layout from the source to the target domains. Our approach outperforms baselines across different settings on multiple large-scale datasets, including adapting across various real city environments, different synthetic sub-domains, from simulated to real environments, and on a novel large-scale dash-cam dataset.

References in corpus (13)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Unsupervised Domain Adaptation by Backpropagation
• Multi-Scale Context Aggregation by Dilated Convolutions
• The Cityscapes Dataset for Semantic Urban Scene Understanding
• Fully Convolutional Networks for Semantic Segmentation
• Learning Deconvolution Network for Semantic Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• Semantic Image Segmentation via Deep Parsing Network
• Playing for Data: Ground Truth from Computer Games
• Instance-aware Semantic Segmentation via Multi-task Network Cascades
• Synthesizing Training Images for Boosting Human 3D Pose Estimation
• Detector Discovery in the Wild: Joint Multiple Instance and Representation Learning

Cited by in corpus (180)

• Deep Visual Domain Adaptation: A Survey
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• The ApolloScape Open Dataset for Autonomous Driving and its Application
• A review of domain adaptation without target labels
• VisDA: The Visual Domain Adaptation Challenge
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Adversarial Learning for Semi-Supervised Semantic Segmentation
• Generalizing to Unseen Domains via Adversarial Data Augmentation
• Patch-based Output Space Adversarial Learning for Joint Optic Disc and Cup Segmentation
• Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation
• Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes
• Self-supervised Domain Adaptation for Computer Vision Tasks
• On Learning Invariant Representation for Domain Adaptation
• Maximum Classifier Discrepancy for Unsupervised Domain Adaptation
• Context-Aware Mixup for Domain Adaptive Semantic Segmentation
• Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation
• Contrastive Adaptation Network for Unsupervised Domain Adaptation
• Unsupervised domain adaptation semantic segmentation of high-resolution remote sensing imagery with invariant domain-level prototype memory
• Complementary Pseudo Labels For Unsupervised Domain Adaptation On Person Re-identification
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
• DAugNet: Unsupervised, Multi-source, Multi-target, and Life-long Domain Adaptation for Semantic Segmentation of Satellite Images
• Domain Stylization: A Strong, Simple Baseline for Synthetic to Real Image Domain Adaptation
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
• How much real data do we actually need: Analyzing object detection performance using synthetic and real data
• Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• Pseudo-Labeling Curriculum for Unsupervised Domain Adaptation
• Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation
• Fully Convolutional Adaptation Networks for Semantic Segmentation
• FDA: Fourier Domain Adaptation for Semantic Segmentation
• One-Sided Unsupervised Domain Mapping
• Label Efficient Learning of Transferable Representations across Domains and Tasks
• Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift
• Adversarial Dropout Regularization
• Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training
• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment
• Stacked Generative Adversarial Networks
• Adversarial Domain Adaptation with Prototype-Based Normalized Output Conditioner
• Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation
• Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach
• Domain Adaptation for Semantic Segmentation with Maximum Squares Loss
• Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning
• Adversarial-Prediction Guided Multi-task Adaptation for Semantic Segmentation of Electron Microscopy Images
• ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks
• Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation
• Exploring Object Relation in Mean Teacher for Cross-Domain Detection
• Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner
• I2F: A Unified Image-to-Feature Approach for Domain Adaptive Semantic Segmentation
• Exploring Categorical Regularization for Domain Adaptive Object Detection
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• Revisiting CycleGAN for semi-supervised segmentation
• Transfer Learning with Label Noise
• FSDR: Frequency Space Domain Randomization for Domain Generalization
• Recognition in Terra Incognita
• SPIGAN: Privileged Adversarial Learning from Simulation
• Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learning
• Image to Image Translation for Domain Adaptation
• Significance-aware Information Bottleneck for Domain Adaptive Semantic Segmentation
• Self-Ensembling GAN for Cross-Domain Semantic Segmentation
• Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation
• No More Discrimination: Cross City Adaptation of Road Scene Segmenters
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
• Open Set Domain Adaptation by Backpropagation
• DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
• What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation
• Semantic Distribution-aware Contrastive Adaptation for Semantic Segmentation
• SceneAdapt: Scene-based domain adaptation for semantic segmentation using adversarial learning
• Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation
• Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data
• Effective Use of Synthetic Data for Urban Scene Semantic Segmentation
• An Adversarial Regularisation for Semi-Supervised Training of Structured Output Neural Networks
• DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception
• Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation
• Boundary and Entropy-driven Adversarial Learning for Fundus Image Segmentation
• Keep it Simple: Image Statistics Matching for Domain Adaptation
• Adaptive Semantic Segmentation with a Strategic Curriculum of Proxy Labels
• Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
• Universal Semi-Supervised Semantic Segmentation
• AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching
• Condition-Invariant Semantic Segmentation
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Multi-Target Domain Adaptation with Collaborative Consistency Learning
• MLAN: Multi-Level Adversarial Network for Domain Adaptive Semantic Segmentation
• Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer
• Style Transfer with Diffusion Models for Synthetic-to-Real Domain Adaptation
• Cross-View Regularization for Domain Adaptive Panoptic Segmentation
• Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers
• Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation
• Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation
• Category Contrast for Unsupervised Domain Adaptation in Visual Tasks
• Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation
• VisDA-2021 Competition Universal Domain Adaptation to Improve Performance on Out-of-Distribution Data
• Semantic-Transferable Weakly-Supervised Endoscopic Lesions Segmentation
• Amplitude Spectrum Transformation for Open Compound Domain Adaptive Semantic Segmentation
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection
• Unsupervised Domain Adaptation for Semantic Segmentation via Low-level Edge Information Transfer
• Semantics-Aware Image to Image Translation and Domain Transfer
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• Domain Adaptation with Auxiliary Target Domain-Oriented Classifier
• Style Normalization and Restitution for Domain Generalization and Adaptation
• Benchmarking the Robustness of Instance Segmentation Models
• Feature-aware Adaptation and Density Alignment for Crowd Counting in Video Surveillance
• Domain Adaptive Medical Image Segmentation via Adversarial Learning of Disease-Specific Spatial Patterns
• Active learning using weakly supervised signals for quality inspection
• DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation
• An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation
• Convex Shape Prior for Deep Neural Convolution Network based Eye Fundus Images Segmentation
• Spatial Attention Pyramid Network for Unsupervised Domain Adaptation
• Domain-adaptive Crowd Counting via High-quality Image Translation and Density Reconstruction
• Adaptation Across Extreme Variations using Unlabeled Domain Bridges
• Exploiting Negative Learning for Implicit Pseudo Label Rectification in Source-Free Domain Adaptive Semantic Segmentation
• Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation
• Weakly-supervised Caricature Face Parsing through Domain Adaptation
• Randomized Histogram Matching: A Simple Augmentation for Unsupervised Domain Adaptation in Overhead Imagery
• Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment
• DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange
• SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning
• Generalizing Deep Models for Overhead Image Segmentation Through Getis-Ord Gi* Pooling
• CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation
• WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation
• Phase Consistent Ecological Domain Adaptation
• MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation
• Context-Aware Domain Adaptation in Semantic Segmentation
• Unsupervised Domain Adaptation: from Simulation Engine to the RealWorld
• Multi-Anchor Active Domain Adaptation for Semantic Segmentation
• CRAVES: Controlling Robotic Arm with a Vision-based Economic System
• Continual Unsupervised Domain Adaptation for Semantic Segmentation
• Unsupervised Classification of Street Architectures Based on InfoGAN
• Light-weight Calibrator: a Separable Component for Unsupervised Domain Adaptation
• Domain Adaptation Meets Disentangled Representation Learning and Style Transfer
• Towards Adaptive Semantic Segmentation by Progressive Feature Refinement
• AutoAdapt: Automated Segmentation Network Search for Unsupervised Domain Adaptation
• Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation
• Learning to simulate complex scenes
• Coarse-to-Fine Domain Adaptive Semantic Segmentation with Photometric Alignment and Category-Center Regularization
• Curriculum CycleGAN for Textual Sentiment Domain Adaptation with Multiple Sources
• LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentation
• DLOW: Domain Flow for Adaptation and Generalization
• Semi-supervised Domain Adaptation for Semantic Segmentation
• Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
• Domain-Adversarial Training of Self-Attention Based Networks for Land Cover Classification using Multi-temporal Sentinel-2 Satellite Imagery
• Two-phase Pseudo Label Densification for Self-training based Domain Adaptation
• Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net
• Transferring and Regularizing Prediction for Semantic Segmentation
• Learn distributed GAN with Temporary Discriminators
• Restyling Data: Application to Unsupervised Domain Adaptation
• A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation
• Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation
• Domain Adaptation on Semantic Segmentation for Aerial Images
• One-shot Unsupervised Domain Adaptation with Personalized Diffusion Models
• Multichannel Semantic Segmentation with Unsupervised Domain Adaptation
• More Separable and Easier to Segment: A Cluster Alignment Method for Cross-Domain Semantic Segmentation
• Deep Neural Networks Regularization for Structured Output Prediction
• Domain Adaptation on Semantic Segmentation with Separate Affine Transformation in Batch Normalization
• Phase Collaborative Network for Two-Phase Medical Image Segmentation
• On Direct Distribution Matching for Adapting Segmentation Networks
• Fast and Accurate Transferability Measurement for Heterogeneous Multivariate Data
• Synthetic training data generation for deep learning based quality inspection
• Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency
• Scale Aware Adaptation for Land-Cover Classification in Remote Sensing Imagery
• Generative AI for Enhanced Wildfire Detection: Bridging the Synthetic-Real Domain Gap
• What Synthesis is Missing: Depth Adaptation Integrated with Weak Supervision for Indoor Scene Parsing
• A Dictionary Approach to Domain-Invariant Learning in Deep Networks
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation
• Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrast
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation
• A Brief Survey and an Application of Semantic Image Segmentation for Autonomous Driving
• Dynamic Adaptation on Non-Stationary Visual Domains
• Leveraging Motion Priors in Videos for Improving Human Segmentation
• On-target Adaptation
• From Real to Synthetic and Back: Synthesizing Training Data for Multi-Person Scene Understanding
• Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentation
• Synthetic Convolutional Features for Improved Semantic Segmentation
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
• Adversarial Domain Randomization
• 3D Scene Parsing via Class-Wise Adaptation
• Adversarial Transfer Learning for Cross-domain Visual Recognition
• CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency