Domain-Adversarial Neural Networks

Abstract

We introduce a new representation learning algorithm suited to the context of domain adaptation, in which data at training and test time come from similar but different distributions. Our algorithm is directly inspired by theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on a data representation that cannot discriminate between the training (source) and test (target) domains. We propose a training objective that implements this idea in the context of a neural network, whose hidden layer is trained to be predictive of the classification task, but uninformative as to the domain of the input. Our experiments on a sentiment analysis classification benchmark, where the target domain data available at training time is unlabeled, show that our neural network for domain adaption algorithm has better performance than either a standard neural network or an SVM, even if trained on input features extracted with the state-of-the-art marginalized stacked denoising autoencoders of Chen et al. (2012).

The first version of this paper was accepted at the "Second Workshop on Transfer and Multi-Task Learning: Theory meets Practice" (NIPS 2014, Montreal, Canada). See: https://sites.google.com/site/multitaskwsnips2014/

References in corpus (3)

• Marginalized Denoising Autoencoders for Domain Adaptation
• Domain Adaptation: Learning Bounds and Algorithms
• Multiple Source Adaptation and the Renyi Divergence

Cited by in corpus (64)

• Learning Transferable Features with Deep Adaptation Networks
• Domain Separation Networks
• Professor Forcing: A New Algorithm for Training Recurrent Networks
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• A Comprehensive Survey of Deep Transfer Learning for Anomaly Detection in Industrial Time Series: Methods, Applications, and Directions
• Learning Adversarially Fair and Transferable Representations
• On Learning Invariant Representation for Domain Adaptation
• Unsupervised Transfer Learning for Anomaly Detection: Application to Complementary Operating Condition Transfer
• Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification
• Adversarial Multi-task Learning for Text Classification
• Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• On The Effects Of Data Normalisation For Domain Adaptation On EEG Data
• Interaction networks for the identification of boosted $H\to b\overline{b}$ decays
• Unsupervised Domain Adversarial Self-Calibration for Electromyographic-based Gesture Recognition
• Toward cross-subject and cross-session generalization in EEG-based emotion recognition: Systematic review, taxonomy, and methods
• Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification
• A Survey on Deep Transfer Learning
• Rethinking Distributional Matching Based Domain Adaptation
• Learning Weighted Representations for Generalization Across Designs
• AD-Cluster: Augmented Discriminative Clustering for Domain Adaptive Person Re-identification
• Adversarial Multi-Criteria Learning for Chinese Word Segmentation
• Invariant Risk Minimization Games
• Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner
• Multiple Source Domain Adaptation with Adversarial Training of Neural Networks
• Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
• AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
• Similarity-preserving Image-image Domain Adaptation for Person Re-identification
• Knowledge Projection for Deep Neural Networks
• Empirical or Invariant Risk Minimization? A Sample Complexity Perspective
• Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation
• Task Specific Adversarial Cost Function
• Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding
• Adversarial Alignment of Class Prediction Uncertainties for Domain Adaptation
• Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
• Physics-Informed Deep Learning and Partial Transfer Learning for Bearing Fault Diagnosis in the Presence of Highly Missing Data
• Study of phase transition of Potts model with Domain Adversarial Neural Network
• Cross Domain Early Crop Mapping using CropSTGAN
• A Theory of Label Propagation for Subpopulation Shift
• DAMIA: Leveraging Domain Adaptation as a Defense against Membership Inference Attacks
• Adversarial Domain Adaptation with Domain Mixup
• Dealing with Nuisance Parameters using Machine Learning in High Energy Physics: a Review
• Camera Adaptation for Fundus-Image-Based CVD Risk Estimation
• Domain Adversarial Reinforcement Learning
• Learning Depth from Monocular Videos Using Synthetic Data: A Temporally-Consistent Domain Adaptation Approach
• Identity Preserving Generative Adversarial Network for Cross-Domain Person Re-identification
• Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions
• DeMIAN: Deep Modality Invariant Adversarial Network
• Principled Hybrids of Generative and Discriminative Domain Adaptation
• Ancient Painting to Natural Image: A New Solution for Painting Processing
• Transfer learning for Remaining Useful Life Prediction Based on Consensus Self-Organizing Models
• Representation Bayesian Risk Decompositions and Multi-Source Domain Adaptation
• Online Monotone Optimization
• Unsupervised Domain Adaptive Learning via Synthetic Data for Person Re-identification
• Beyond the Deep Metric Learning: Enhance the Cross-Modal Matching with Adversarial Discriminative Domain Regularization
• Multi-Purposing Domain Adaptation Discriminators for Pseudo Labeling Confidence
• StRDAN: Synthetic-to-Real Domain Adaptation Network for Vehicle Re-Identification
• Metric-Learning-Assisted Domain Adaptation
• Learning Purified Feature Representations from Task-irrelevant Labels
• Domain-Adversarial Multi-Task Framework for Novel Therapeutic Property Prediction of Compounds
• Self-Guided Adaptation: Progressive Representation Alignment for Domain Adaptive Object Detection
• KL Guided Domain Adaptation
• Towards All-around Knowledge Transferring: Learning From Task-irrelevant Labels
• Domain Adaptation for Real-World Single View 3D Reconstruction