ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation

Abstract

Semantic segmentation is a key problem for many computer vision tasks. While approaches based on convolutional neural networks constantly break new records on different benchmarks, generalizing well to diverse testing environments remains a major challenge. In numerous real world applications, there is indeed a large gap between data distributions in train and test domains, which results in severe performance loss at run-time. In this work, we address the task of unsupervised domain adaptation in semantic segmentation with losses based on the entropy of the pixel-wise predictions. To this end, we propose two novel, complementary methods using (i) entropy loss and (ii) adversarial loss respectively. We demonstrate state-of-the-art performance in semantic segmentation on two challenging "synthetic-2-real" set-ups and show that the approach can also be used for detection.

Accepted in CVPR'19. Code is available at https://github.com/valeoai/ADVENT

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation by Backpropagation
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• Temporal Ensembling for Semi-Supervised Learning
• Unsupervised Domain Adaptation with Residual Transfer Networks
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Domain Adaptation for Visual Applications: A Comprehensive Survey

Cited by in corpus (187)

• A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images
• Self-supervised Domain Adaptation for Computer Vision Tasks
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation
• Night-time Scene Parsing with a Large Real Dataset
• Context-Aware Mixup for Domain Adaptive Semantic Segmentation
• Enabling Country-Scale Land Cover Mapping with Meter-Resolution Satellite Imagery
• Multi-Site Infant Brain Segmentation Algorithms: The iSeg-2019 Challenge
• Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation
• Unsupervised Scene Adaptation with Memory Regularization in vivo
• Addressing Failure Prediction by Learning Model Confidence
• Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data
• Unsupervised domain adaptation semantic segmentation of high-resolution remote sensing imagery with invariant domain-level prototype memory
• LE-UDA: Label-efficient unsupervised domain adaptation for medical image segmentation
• Margin Preserving Self-paced Contrastive Learning Towards Domain Adaptation for Medical Image Segmentation
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• Self-paced and self-consistent co-training for semi-supervised image segmentation
• Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation
• Generative Domain Adaptation for Face Anti-Spoofing
• Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation
• Source-Relaxed Domain Adaptation for Image Segmentation
• Adversarial Shape Learning for Building Extraction in VHR Remote Sensing Images
• Domain adaptation based self-correction model for COVID-19 infection segmentation in CT images
• FDA: Fourier Domain Adaptation for Semantic Segmentation
• A Survey on Deep Domain Adaptation for LiDAR Perception
• Affinity Space Adaptation for Semantic Segmentation Across Domains
• IDDA: a large-scale multi-domain dataset for autonomous driving
• PSIGAN: Joint probabilistic segmentation and image distribution matching for unpaired cross-modality adaptation based MRI segmentation
• Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment
• Adversarial Domain Adaptation with Prototype-Based Normalized Output Conditioner
• Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision
• Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformation
• Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach
• Boosting Night-time Scene Parsing with Learnable Frequency
• Domain Adaptation for Semantic Segmentation with Maximum Squares Loss
• MT-UDA: Towards Unsupervised Cross-modality Medical Image Segmentation with Limited Source Labels
• Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation
• Improving Semantic Segmentation via Self-Training
• PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation
• Exploring Feature Representation Learning for Semi-supervised Medical Image Segmentation
• Confidence Regularized Self-Training
• Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation
• Unsupervised domain adaptation based COVID-19 CT infection segmentation network
• Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation
• I2F: A Unified Image-to-Feature Approach for Domain Adaptive Semantic Segmentation
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• FSDR: Frequency Space Domain Randomization for Domain Generalization
• Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations
• Domain Adaptation for Semantic Segmentation via Patch-Wise Contrastive Learning
• Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
• A Prototype-Oriented Framework for Unsupervised Domain Adaptation
• Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency
• Self-Ensembling GAN for Cross-Domain Semantic Segmentation
• Multiple instance learning on deep features for weakly supervised object detection with extreme domain shifts
• Birds of A Feather Flock Together: Category-Divergence Guidance for Domain Adaptive Segmentation
• Boosting Semi-supervised Image Segmentation with Global and Local Mutual Information Regularization
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation
• What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation
• Semantic Distribution-aware Contrastive Adaptation for Semantic Segmentation
• Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models
• Instance Adaptive Self-Training for Unsupervised Domain Adaptation
• Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation
• Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation
• Unsupervised Continual Semantic Adaptation through Neural Rendering
• DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort
• Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error
• Boundary and Entropy-driven Adversarial Learning for Fundus Image Segmentation
• DDF: A Novel Dual-Domain Image Fusion Strategy for Remote Sensing Image Semantic Segmentation with Unsupervised Domain Adaptation
• AdaSemiCD: An Adaptive Semi-Supervised Change Detection Method Based on Pseudo-Label Evaluation
• Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond
• Condition-Invariant Semantic Segmentation
• AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching
• Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization
• Multi-Target Domain Adaptation with Collaborative Consistency Learning
• MLAN: Multi-Level Adversarial Network for Domain Adaptive Semantic Segmentation
• Domain Adaptation Through Task Distillation
• Source-Free Domain Adaptation for Semantic Segmentation
• Multi-source Pseudo-label Learning of Semantic Segmentation for the Scene Recognition of Agricultural Mobile Robots
• Test time Adaptation through Perturbation Robustness
• Cross-View Regularization for Domain Adaptive Panoptic Segmentation
• Simulations of Common Unsupervised Domain Adaptation Algorithms for Image Classification
• Self-Adversarial Disentangling for Specific Domain Adaptation
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation
• Multi-Source Domain Adaptation with Collaborative Learning for Semantic Segmentation
• Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation
• Category Contrast for Unsupervised Domain Adaptation in Visual Tasks
• Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels
• RarePlanes: Synthetic Data Takes Flight
• PIG: Prompt Images Guidance for Night-Time Scene Parsing
• Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation
• Amplitude Spectrum Transformation for Open Compound Domain Adaptive Semantic Segmentation
• Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding
• Style Normalization and Restitution for Domain Generalization and Adaptation
• Unified Domain Adaptive Semantic Segmentation
• Unsupervised Domain Adaptation for Semantic Segmentation via Low-level Edge Information Transfer
• Active learning using weakly supervised signals for quality inspection
• ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities
• Distribution Regularized Self-Supervised Learning for Domain Adaptation of Semantic Segmentation
• Pseudolabel guided pixels contrast for domain adaptive semantic segmentation
• An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation
• DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation
• Entropy Guided Adversarial Model for Weakly Supervised Object Localization
• Bi-Mix: Bidirectional Mixing for Domain Adaptive Nighttime Semantic Segmentation
• Exploiting Negative Learning for Implicit Pseudo Label Rectification in Source-Free Domain Adaptive Semantic Segmentation
• Cross Branch Feature Fusion Decoder for Consistency Regularization-based Semi-Supervised Change Detection
• BIM Hyperreality: Data Synthesis Using BIM and Hyperrealistic Rendering for Deep Learning
• Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation
• Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
• Collaborative Training between Region Proposal Localization and Classification for Domain Adaptive Object Detection
• Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment
• Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation
• SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning
• Unsupervised domain adaptation via coarse-to-fine feature alignment method using contrastive learning
• Bi-Classifier Determinacy Maximization for Unsupervised Domain Adaptation
• Uncertainty-Aware Unsupervised Domain Adaptation in Object Detection
• Improving Road Segmentation in Challenging Domains Using Similar Place Priors
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study
• Unsupervised Domain Adaptation for Video Semantic Segmentation
• Channel-wise Alignment for Adaptive Object Detection
• Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting
• CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation
• Union-set Multi-source Model Adaptation for Semantic Segmentation
• Phase Consistent Ecological Domain Adaptation
• WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation
• Cross-Region Domain Adaptation for Class-level Alignment
• ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation
• Data-Efficient Framework for Real-world Multiple Sound Source 2D Localization
• Continual Unsupervised Domain Adaptation for Semantic Segmentation
• Learning Cross-modal Contrastive Features for Video Domain Adaptation
• Covariance-aware Feature Alignment with Pre-computed Source Statistics for Test-time Adaptation to Multiple Image Corruptions
• Automatic Validation of Textual Attribute Values in E-commerce Catalog by Learning with Limited Labeled Data
• Multi-Anchor Active Domain Adaptation for Semantic Segmentation
• Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling
• Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentation
• Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation
• Coarse-to-Fine Domain Adaptive Semantic Segmentation with Photometric Alignment and Category-Center Regularization
• Energy-constrained Self-training for Unsupervised Domain Adaptation
• ECAP: Extensive Cut-and-Paste Augmentation for Unsupervised Domain Adaptive Semantic Segmentation
• Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction
• AutoAdapt: Automated Segmentation Network Search for Unsupervised Domain Adaptation
• Generalizable Model-agnostic Semantic Segmentation via Target-specific Normalization
• Cross-Domain Transfer Learning with CoRTe: Consistent and Reliable Transfer from Black-Box to Lightweight Segmentation Model
• Synergistic saliency and depth prediction for RGB-D saliency detection
• Target Consistency for Domain Adaptation: when Robustness meets Transferability
• Cross-denoising Network against Corrupted Labels in Medical Image Segmentation with Domain Shift
• Transferring and Regularizing Prediction for Semantic Segmentation
• Two-phase Pseudo Label Densification for Self-training based Domain Adaptation
• Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation
• Semi-supervised Domain Adaptation for Semantic Segmentation
• LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentation
• DA-BEV: Unsupervised Domain Adaptation for Bird's Eye View Perception
• ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods
• Spectral Unsupervised Domain Adaptation for Visual Recognition
• Synthetic training data generation for deep learning based quality inspection
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation
• More Separable and Easier to Segment: A Cluster Alignment Method for Cross-Domain Semantic Segmentation
• TridentAdapt: Learning Domain-invariance via Source-Target Confrontation and Self-induced Cross-domain Augmentation
• Diversified Multi-prototype Representation for Semi-supervised Segmentation
• One-shot Unsupervised Domain Adaptation with Personalized Diffusion Models
• Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrast
• Generative AI for Enhanced Wildfire Detection: Bridging the Synthetic-Real Domain Gap
• Mutual-GAN: Towards Unsupervised Cross-Weather Adaptation with Mutual Information Constraint
• Self-Adaptive Partial Domain Adaptation
• Domain Adaptation on Semantic Segmentation with Separate Affine Transformation in Batch Normalization
• Exploiting Inter-pixel Correlations in Unsupervised Domain Adaptation for Semantic Segmentation
• Ensemble of Discriminators for Domain Adaptation in Multiple Sound Source 2D Localization
• Domain Adaptation on Semantic Segmentation for Aerial Images
• Unsupervised Domain Adaptation Network with Category-Centric Prototype Aligner for Biomedical Image Segmentation
• On Direct Distribution Matching for Adapting Segmentation Networks
• Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentation
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation
• MetaPix: Domain Transfer for Semantic Segmentation by Meta Pixel Weighting
• A Teacher-Student Framework with Fourier Augmentation for COVID-19 Infection Segmentation in CT Images
• Domain Adaptive Semantic Segmentation without Source Data
• Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval
• Learning with Weak Annotations for Robust Maritime Obstacle Detection
• Consistency Regularization with High-dimensional Non-adversarial Source-guided Perturbation for Unsupervised Domain Adaptation in Segmentation
• Reimagine BiSeNet for Real-Time Domain Adaptation in Semantic Segmentation
• Adapt Everywhere: Unsupervised Adaptation of Point-Clouds and Entropy Minimisation for Multi-modal Cardiac Image Segmentation
• DAVOS: Semi-Supervised Video Object Segmentation via Adversarial Domain Adaptation
• Towards Uncovering the Intrinsic Data Structures for Unsupervised Domain Adaptation using Structurally Regularized Deep Clustering
• Context-aware virtual adversarial training for anatomically-plausible segmentation
• Leveraging Domain Adaptation for Low-Resource Geospatial Machine Learning
• ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation
• WeClick: Weakly-Supervised Video Semantic Segmentation with Click Annotations