Domain-Adversarial Training of Neural Networks

Abstract

We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains. The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation and stochastic gradient descent, and can thus be implemented with little effort using any of the deep learning packages. We demonstrate the success of our approach for two distinct classification problems (document sentiment analysis and image classification), where state-of-the-art domain adaptation performance on standard benchmarks is achieved. We also validate the approach for descriptor learning task in the context of person re-identification application.

Published in JMLR: http://jmlr.org/papers/v17/15-239.html

Cited by in corpus (23)

• Discriminative Adversarial Domain Generalization with Meta-learning based Cross-domain Validation
• A joint model of unpaired data from scRNA-seq and spatial transcriptomics for imputing missing gene expression measurements
• Domain-Symmetric Networks for Adversarial Domain Adaptation
• Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
• From exploration to control: learning object manipulation skills through novelty search and local adaptation
• A survey on Adversarial Recommender Systems: from Attack/Defense strategies to Generative Adversarial Networks
• Learning Pose-invariant 3D Object Reconstruction from Single-view Images
• Domain Adaptive Adversarial Learning Based on Physics Model Feedback for Underwater Image Enhancement
• Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings
• Unsupervised Domain Adaptation Learning Algorithm for RGB-D Staircase Recognition
• Adversarial Cross-Domain Action Recognition with Co-Attention
• DAMIA: Leveraging Domain Adaptation as a Defense against Membership Inference Attacks
• Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data
• Geodesics in fibered latent spaces: A geometric approach to learning correspondences between conditions
• Skin Deep Unlearning: Artefact and Instrument Debiasing in the Context of Melanoma Classification
• Self-Supervised Dynamic Networks for Covariate Shift Robustness
• Fast and Accurate Transferability Measurement for Heterogeneous Multivariate Data
• CAPE: Context-Aware Private Embeddings for Private Language Learning
• DWMD: Dimensional Weighted Orderwise Moment Discrepancy for Domain-specific Hidden Representation Matching
• Decentralized Deep Reinforcement Learning for Network Level Traffic Signal Control
• Domain Adaptation for Rare Classes Augmented with Synthetic Samples
• Generative Adversarial Networks for Video-to-Video Domain Adaptation
• Efficient Variational Graph Autoencoders for Unsupervised Cross-domain Prerequisite Chains