Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks

Abstract

Collecting well-annotated image datasets to train modern machine learning algorithms is prohibitively expensive for many tasks. One appealing alternative is rendering synthetic data where ground-truth annotations are generated automatically. Unfortunately, models trained purely on rendered images often fail to generalize to real images. To address this shortcoming, prior work introduced unsupervised domain adaptation algorithms that attempt to map representations between the two domains or learn to extract features that are domain-invariant. In this work, we present a new approach that learns, in an unsupervised manner, a transformation in the pixel space from one domain to the other. Our generative adversarial network (GAN)-based method adapts source-domain images to appear as if drawn from the target domain. Our approach not only produces plausible samples, but also outperforms the state-of-the-art on a number of unsupervised domain adaptation scenarios by large margins. Finally, we demonstrate that the adaptation process generalizes to object classes unseen during training.

arXiv commentFinal CVPR 2017 paper and supplementary material

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Learning Transferable Features with Deep Adaptation Networks
• Deep Domain Confusion: Maximizing for Domain Invariance
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution
• Domain Separation Networks
• Domain-Adversarial Neural Networks
• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model
• Playing for Data: Ground Truth from Computer Games
• Coupled Generative Adversarial Networks
• ResearchDoom and CocoDoom: Learning Computer Vision with Games
• UnrealCV: Connecting Computer Vision to Unreal Engine

Cited by in corpus (37)

• Generative Adversarial Networks: An Overview
• A DIRT-T Approach to Unsupervised Domain Adaptation
• Domain Adaptation for Visual Applications: A Comprehensive Survey
• Cross-view Semantic Segmentation for Sensing Surroundings
• An Introduction to Image Synthesis with Generative Adversarial Nets
• One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
• Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation
• Label Efficient Learning of Transferable Representations across Domains and Tasks
• Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification
• Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift
• Is Generator Conditioning Causally Related to GAN Performance?
• Towards Large-Pose Face Frontalization in the Wild
• Associative Domain Adaptation
• Camera Style Adaptation for Person Re-identification
• Knowledge Distillation Methods for Efficient Unsupervised Adaptation Across Multiple Domains
• Significance-aware Information Bottleneck for Domain Adaptive Semantic Segmentation
• DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation
• Importance Weighted Adversarial Nets for Partial Domain Adaptation
• Synthetic to Real Adaptation with Generative Correlation Alignment Networks
• ACE: Adapting to Changing Environments for Semantic Segmentation
• Revealing Hidden Potentials of the q-Space Signal in Breast Cancer
• DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)
• Domain Alignment with Triplets
• A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes
• A Unified Framework for Generalizable Style Transfer: Style and Content Separation
• Learning Agile Locomotion via Adversarial Training
• Beyond Domain Adaptation: Unseen Domain Encapsulation via Universal Non-volume Preserving Models
• CRAFT: Complementary Recommendations Using Adversarial Feature Transformer
• Identity Preserving Generative Adversarial Network for Cross-Domain Person Re-identification
• Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagery
• Unsupervised Domain Adaptation using Generative Models and Self-ensembling
• Visual Data Augmentation through Learning
• Guiding the One-to-one Mapping in CycleGAN via Optimal Transport
• Taming Adversarial Domain Transfer with Structural Constraints for Image Enhancement
• (DE)^2 CO: Deep Depth Colorization
• Image to Video Domain Adaptation Using Web Supervision
• Parameter Reference Loss for Unsupervised Domain Adaptation