Unsupervised Domain Adaptation by Backpropagation

Abstract

Top-performing deep architectures are trained on massive amounts of labeled data. In the absence of labeled data for a certain task, domain adaptation often provides an attractive option given that labeled data of similar nature but from a different domain (e.g. synthetic images) are available. Here, we propose a new approach to domain adaptation in deep architectures that can be trained on large amount of labeled data from the source domain and large amount of unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of "deep" features that are (i) discriminative for the main learning task on the source domain and (ii) invariant with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a simple new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation. Overall, the approach can be implemented with little effort using any of the deep-learning packages. The method performs very well in a series of image classification experiments, achieving adaptation effect in the presence of big domain shifts and outperforming previous state-of-the-art on Office datasets.

References in corpus (5)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Deep Domain Confusion: Maximizing for Domain Invariance
• Domain Adaptive Neural Networks for Object Recognition
• One-Shot Adaptation of Supervised Deep Convolutional Models

Cited by in corpus (209)

• Deep Visual Domain Adaptation: A Survey
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation
• Deep Subdomain Adaptation Network for Image Classification
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• A review of domain adaptation without target labels
• Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
• Asymmetric Tri-training for Unsupervised Domain Adaptation
• Adversarial Discriminative Domain Adaptation
• Aligning Domain-specific Distribution and Classifier for Cross-domain Classification from Multiple Sources
• Maximum Density Divergence for Domain Adaptation
• A Survey of End-to-End Driving: Architectures and Training Methods
• Multi-Representation Adaptation Network for Cross-domain Image Classification
• Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts
• Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes
• Deep Transfer Learning for Person Re-identification
• What, Where and How to Transfer in SAR Target Recognition Based on Deep CNNs
• Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis
• A Survey of mmWave-based Human Sensing: Technology, Platform and Applications
• Deep Residual Correction Network for Partial Domain Adaptation
• Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild
• Silent Speech Interfaces for Speech Restoration: A Review
• Maximum Classifier Discrepancy for Unsupervised Domain Adaptation
• DGCN: Diversified Recommendation with Graph Convolutional Networks
• Learning Disentangled Semantic Representation for Domain Adaptation
• Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain
• Cluster-guided Asymmetric Contrastive Learning for Unsupervised Person Re-Identification
• Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap
• Speaker-Invariant Training via Adversarial Learning
• Review: Deep Learning in Electron Microscopy
• FoodAI: Food Image Recognition via Deep Learning for Smart Food Logging
• Deep Domain Adaptation Based Video Smoke Detection using Synthetic Smoke Images
• Learning Disentangled Representations in the Imaging Domain
• Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification
• Causally Regularized Learning with Agnostic Data Selection Bias
• Cross-regional oil palm tree counting and detection via multi-level attention domain adaptation network
• Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
• Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars
• Knowledge Transfer between Buildings for Seismic Damage Diagnosis through Adversarial Learning
• Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview
• Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation
• Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation
• Cross-Domain Complementary Learning Using Pose for Multi-Person Part Segmentation
• Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognition
• Generalized Domain Conditioned Adaptation Network
• Missing-Class-Robust Domain Adaptation by Unilateral Alignment for Fault Diagnosis
• Search for direct top squark pair production in events with one lepton, jets, and missing transverse momentum at 13 TeV with the CMS experiment
• Meta Balanced Network for Fair Face Recognition
• Synthesizing Training Images for Boosting Human 3D Pose Estimation
• Adversarial Speaker Verification
• Sliding-Window Normalization to Improve the Performance of Machine-Learning Models for Real-Time Motion Prediction Using Electromyography
• Domain Adversarial Spatial-Temporal Network: A Transferable Framework for Short-term Traffic Forecasting across Cities
• TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation
• Multi-modal Deep Analysis for Multimedia
• Cycle-Consistent Speech Enhancement
• A Survey of Deep Learning for Scientific Discovery
• A deep neural network to search for new long-lived particles decaying to jets
• Uncertainty Aware Learning for High Energy Physics
• Dynamic Instance Domain Adaptation
• Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift
• Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation
• Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification
• Adversarial Dropout Regularization
• Vertex and Energy Reconstruction in JUNO with Machine Learning Methods
• Cross-Market Product Recommendation
• Unlearning Protected User Attributes in Recommendations with Adversarial Training
• No-reference Screen Content Image Quality Assessment with Unsupervised Domain Adaptation
• Relation Matters: Foreground-aware Graph-based Relational Reasoning for Domain Adaptive Object Detection
• Adversarial Domain Adaptation with Prototype-Based Normalized Output Conditioner
• PoreNet: CNN-based Pore Descriptor for High-resolution Fingerprint Recognition
• Learning Target Domain Specific Classifier for Partial Domain Adaptation
• Speaker Adaptation for Attention-Based End-to-End Speech Recognition
• Adversarial Speaker Adaptation
• MT-UDA: Towards Unsupervised Cross-modality Medical Image Segmentation with Limited Source Labels
• Domain Adaptive Person Re-Identification via Coupling Optimization
• Robust Domain Misinformation Detection via Multi-modal Feature Alignment
• Privacy-preserving Federated Adversarial Domain Adaption over Feature Groups for Interpretability
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation
• A Noise-Robust Self-supervised Pre-training Model Based Speech Representation Learning for Automatic Speech Recognition
• PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation
• Dark sector searches with the CMS experiment
• 3D Human Pose Estimation in the Wild by Adversarial Learning
• From A to Z: Supervised Transfer of Style and Content Using Deep Neural Network Generators
• Regularizing Neural Network Training via Identity-wise Discriminative Feature Suppression
• Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective
• An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training
• Exploring Categorical Regularization for Domain Adaptive Object Detection
• Outside the Box: Abstraction-Based Monitoring of Neural Networks
• Multi-source Domain Adaptation for Text-independent Forensic Speaker Recognition
• An Empirical Study on Channel Effects for Synthetic Voice Spoofing Countermeasure Systems
• Knowledge Distillation Methods for Efficient Unsupervised Adaptation Across Multiple Domains
• Search for top squarks in final states with two top quarks and several light-flavor jets in proton-proton collisions at $\sqrt{s}=$ 13 TeV
• Image to Image Translation for Domain Adaptation
• DVR: Micro-Video Recommendation Optimizing Watch-Time-Gain under Duration Bias
• Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation
• Continuous Gesture Recognition from sEMG Sensor Data with Recurrent Neural Networks and Adversarial Domain Adaptation
• Harmonizing Transferability and Discriminability for Adapting Object Detectors
• No More Discrimination: Cross City Adaptation of Road Scene Segmenters
• Uncertainty-Guided Mixup for Semi-Supervised Domain Adaptation without Source Data
• Semi-supervised representation learning via dual autoencoders for domain adaptation
• Beyond Ads: Sequential Decision-Making Algorithms in Law and Public Policy
• Adaptive Betweenness Clustering for Semi-Supervised Domain Adaptation
• Daft-Exprt: Cross-Speaker Prosody Transfer on Any Text for Expressive Speech Synthesis
• Importance Weighted Adversarial Nets for Partial Domain Adaptation
• All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentation
• Coupling Adversarial Learning with Selective Voting Strategy for Distribution Alignment in Partial Domain Adaptation
• DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception
• Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs
• A Dataset and Benchmark Towards Multi-Modal Face Anti-Spoofing Under Surveillance Scenarios
• Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling
• Unsupervised Real-world Image Super Resolution via Domain-distance Aware Training
• Semi-Supervised Disentangled Framework for Transferable Named Entity Recognition
• Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision
• Prototypical Partial Optimal Transport for Universal Domain Adaptation
• Disentangled speaker and nuisance attribute embedding for robust speaker verification
• A Flexible Selection Scheme for Minimum-Effort Transfer Learning
• Heterogeneous Face Frontalization via Domain Agnostic Learning
• Adaptive Semantic Segmentation with a Strategic Curriculum of Proxy Labels
• Source Free Unsupervised Graph Domain Adaptation
• Distributed Learning and Inference with Compressed Images
• Active Learning to Classify Macromolecular Structures in situ for Less Supervision in Cryo-Electron Tomography
• POEM: Polarization of Embeddings for Domain-Invariant Representations
• Controlling for Confounders in Multimodal Emotion Classification via Adversarial Learning
• Sensor-invariant Fingerprint ROI Segmentation Using Recurrent Adversarial Learning
• Improving Domain Adaptation Through Class Aware Frequency Transformation
• Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge Transfer
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Scalable Greedy Algorithms for Transfer Learning
• Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks
• EllSeg-Gen, towards Domain Generalization for head-mounted eyetracking
• Semantic Segmentation in Multiple Adverse Weather Conditions with Domain Knowledge Retention
• Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation
• A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds
• Cryo-shift: Reducing domain shift in cryo-electron subtomograms with unsupervised domain adaptation and randomization
• Improving Automated Sonar Video Analysis to Notify About Jellyfish Blooms
• Attentive Adversarial Learning for Domain-Invariant Training
• Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network
• Domain Alignment with Triplets
• Improving Compound Activity Classification via Deep Transfer and Representation Learning
• Learning High Dynamic Range from Outdoor Panoramas
• Transfering Low-Frequency Features for Domain Adaptation
• Cut-and-Paste Dataset Generation for Balancing Domain Gaps in Object Instance Detection
• Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation
• VisDA-2021 Competition Universal Domain Adaptation to Improve Performance on Out-of-Distribution Data
• Unsupervised Domain Adaptation for Cross-sensor Pore Detection in High-resolution Fingerprint Images
• Micro-video Tagging via Jointly Modeling Social Influence and Tag Relation
• Modeling relation paths for knowledge base completion via joint adversarial training
• A Multi Camera Unsupervised Domain Adaptation Pipeline for Object Detection in Cultural Sites through Adversarial Learning and Self-Training
• Improving Transferability of Domain Adaptation Networks Through Domain Alignment Layers
• Learning Invariant Representation for Unsupervised Image Restoration
• Adaptive Domain Generalization via Online Disagreement Minimization
• Adversarial Bi-Regressor Network for Domain Adaptive Regression
• An Adversarial Domain Separation Framework for Septic Shock Early Prediction Across EHR Systems
• Representation Learning for Audio Privacy Preservation using Source Separation and Robust Adversarial Learning
• Predictive Uncertainty-based Bias Mitigation in Ranking
• Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation
• Emo-DNA: Emotion Decoupling and Alignment Learning for Cross-Corpus Speech Emotion Recognition
• FAN-Net: Fourier-Based Adaptive Normalization For Cross-Domain Stroke Lesion Segmentation
• Small Towers Make Big Differences
• Domain Adversarial Reinforcement Learning
• DADNN: Multi-Scene CTR Prediction via Domain-Aware Deep Neural Network
• KS(conf ): A Light-Weight Test if a ConvNet Operates Outside of Its Specifications
• Uncertainty-Autoencoder-Based Privacy and Utility Preserving Data Type Conscious Transformation
• Adaptive Object Detection with Dual Multi-Label Prediction
• Learning to Exploit Multiple Vision Modalities by Using Grafted Networks
• Domain Adversarial RetinaNet as a Reference Algorithm for the MItosis DOmain Generalization Challenge
• Alioth: A Machine Learning Based Interference-Aware Performance Monitor for Multi-Tenancy Applications in Public Cloud
• A Theory of Output-Side Unsupervised Domain Adaptation
• A Low-Cost Neural ODE with Depthwise Separable Convolution for Edge Domain Adaptation on FPGAs
• Optimal MRI Undersampling Patterns for Ultimate Benefit of Medical Vision Tasks
• CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
• BiPC: Bidirectional Probability Calibration for Unsupervised Domain Adaption
• Delving into Robust Object Detection from Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach
• Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery
• Disentangling representations of retinal images with generative models
• Identity Preserving Generative Adversarial Network for Cross-Domain Person Re-identification
• The Rediscovery Hypothesis: Language Models Need to Meet Linguistics
• Unsupervised Domain Adaptation for Spatio-Temporal Action Localization
• DeMIAN: Deep Modality Invariant Adversarial Network
• Union-set Multi-source Model Adaptation for Semantic Segmentation
• Data Priming Network for Automatic Check-Out
• An Attribute-Aligned Strategy for Learning Speech Representation
• Self-balanced Learning For Domain Generalization
• CUDA: Contradistinguisher for Unsupervised Domain Adaptation
• Adapting a FrameNet Semantic Parser for Spoken Language Understanding Using Adversarial Learning
• The Unreasonable Effectiveness of Large Language-Vision Models for Source-free Video Domain Adaptation
• MULE: Multimodal Universal Language Embedding
• Open Compound Domain Adaptation
• I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting
• OmniPrint: A Configurable Printed Character Synthesizer
• Transfer Learning for Aided Target Recognition: Comparing Deep Learning to other Machine Learning Approaches
• Unsupervised Multi-Domain Multimodal Image-to-Image Translation with Explicit Domain-Constrained Disentanglement
• Multimodal Deep Domain Adaptation
• Spatial-Aware GAN for Unsupervised Person Re-identification
• AdaVocoder: Adaptive Vocoder for Custom Voice
• G2R Bound: A Generalization Bound for Supervised Learning from GAN-Synthetic Data
• StRDAN: Synthetic-to-Real Domain Adaptation Network for Vehicle Re-Identification
• Multichannel Semantic Segmentation with Unsupervised Domain Adaptation
• Dual Domain-Adversarial Learning for Audio-Visual Saliency Prediction
• Beyond Photo Realism for Domain Adaptation from Synthetic Data
• Test-Time Adaptation with Principal Component Analysis
• Synthesizing Photorealistic Images with Deep Generative Learning
• Learning Deep Features for Shape Correspondence with Domain Invariance
• Leveraging Motion Priors in Videos for Improving Human Segmentation
• Learning Less Generalizable Patterns with an Asymmetrically Trained Double Classifier for Better Test-Time Adaptation
• The Role of Embedding Complexity in Domain-invariant Representations
• Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization
• Parameter Reference Loss for Unsupervised Domain Adaptation