DualGAN: Unsupervised Dual Learning for Image-to-Image Translation

Abstract

Conditional Generative Adversarial Networks (GANs) for cross-domain image-to-image translation have made much progress recently. Depending on the task complexity, thousands to millions of labeled image pairs are needed to train a conditional GAN. However, human labeling is expensive, even impractical, and large quantities of data may not always be available. Inspired by dual learning from natural language translation, we develop a novel dual-GAN mechanism, which enables image translators to be trained from two sets of unlabeled images from two domains. In our architecture, the primal GAN learns to translate images from domain U to those in domain V, while the dual GAN learns to invert the task. The closed loop made by the primal and dual tasks allows images from either domain to be translated and then reconstructed. Hence a loss function that accounts for the reconstruction error of images can be used to train the translators. Experiments on multiple image translation tasks with unlabeled data show considerable performance gain of DualGAN over a single GAN. For some tasks, DualGAN can even achieve comparable or slightly better results than conditional GAN trained on fully labeled data.

Accepted by ICCV 2017

Cited by in corpus (84)

• Deep Visual Domain Adaptation: A Survey
• Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
• Multimodal Unsupervised Image-to-Image Translation
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
• One-Shot Unsupervised Cross Domain Translation
• CariGANs: Unpaired Photo-to-Caricature Translation
• Generative Models for Automatic Chemical Design
• Triangle Generative Adversarial Networks
• Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption
• One-Sided Unsupervised Domain Mapping
• Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification
• GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data
• StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
• Exemplar Guided Unsupervised Image-to-Image Translation with Semantic Consistency
• Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
• Conditional Generation of Medical Images via Disentangled Adversarial Inference
• ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes
• Unsupervised Cipher Cracking Using Discrete GANs
• JointGAN: Multi-Domain Joint Distribution Learning with Generative Adversarial Nets
• Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Human-Like Sketches
• Disentangled Makeup Transfer with Generative Adversarial Network
• Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks
• Conditional Image-to-Image Translation
• CariGAN: Caricature Generation through Weakly Paired Adversarial Learning
• Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
• Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation
• Non-Adversarial Unsupervised Word Translation
• Learning to Sketch with Shortcut Cycle Consistency
• Multi-Domain Translation by Learning Uncoupled Autoencoders
• UGAN: Untraceable GAN for Multi-Domain Face Translation
• Discriminative Region Proposal Adversarial Networks for High-Quality Image-to-Image Translation
• MISO: Mutual Information Loss with Stochastic Style Representations for Multimodal Image-to-Image Translation
• IterGANs: Iterative GANs to Learn and Control 3D Object Transformation
• Synthetic CT Generation from MRI Using Improved DualGAN
• Face-to-Parameter Translation for Game Character Auto-Creation
• T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks
• Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation
• Unsupervised Single Image Deraining with Self-supervised Constraints
• Learning from Multi-domain Artistic Images for Arbitrary Style Transfer
• Unsupervised Speech Domain Adaptation Based on Disentangled Representation Learning for Robust Speech Recognition
• GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
• Mocycle-GAN: Unpaired Video-to-Video Translation
• Handloom Design Generation Using Generative Networks
• Learning Compositional Visual Concepts with Mutual Consistency
• Learning Disentangling and Fusing Networks for Face Completion Under Structured Occlusions
• Language-Driven Image Style Transfer
• SDIT: Scalable and Diverse Cross-domain Image Translation
• MTS-CycleGAN: An Adversarial-based Deep Mapping Learning Network for Multivariate Time Series Domain Adaptation Applied to the Ironmaking Industry
• Attention-GAN for Object Transfiguration in Wild Images
• Unpaired Photo-to-Caricature Translation on Faces in the Wild
• Identity Preserving Generative Adversarial Network for Cross-Domain Person Re-identification
• EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer
• Texture Deformation Based Generative Adversarial Networks for Face Editing
• ReLGAN: Generalization of Consistency for GAN with Disjoint Constraints and Relative Learning of Generative Processes for Multiple Transformation Learning
• Training Generative Adversarial Networks with Adaptive Composite Gradient
• Deep Learning with Inaccurate Training Data for Image Restoration
• Domain-Specific Mappings for Generative Adversarial Style Transfer
• Unsupervised Image Super-Resolution with an Indirect Supervised Path
• GANtruth - an unpaired image-to-image translation method for driving scenarios
• Deep Consensus Learning
• Synthetic Dynamic PMU Data Generation: A Generative Adversarial Network Approach
• Total Generate: Cycle in Cycle Generative Adversarial Networks for Generating Human Faces, Hands, Bodies, and Natural Scenes
• Generative Creativity: Adversarial Learning for Bionic Design
• Memory-guided Unsupervised Image-to-image Translation
• Excavate Condition-invariant Space by Intrinsic Encoder
• GANILLA: Generative Adversarial Networks for Image to Illustration Translation
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition
• Improved Surrogates in Inertial Confinement Fusion with Manifold and Cycle Consistencies
• Quality-aware Unpaired Image-to-Image Translation
• Synthesizing Photorealistic Images with Deep Generative Learning
• Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain
• One to Multiple Mapping Dual Learning: Learning Multiple Sources from One Mixed Signal
• Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
• Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks
• Variational learning across domains with triplet information
• Generative Adversarial Networks for Video-to-Video Domain Adaptation
• Deep Factorised Inverse-Sketching
• GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer
• Association: Remind Your GAN not to Forget
• PCGAN: Partition-Controlled Human Image Generation
• BSD-GAN: Branched Generative Adversarial Network for Scale-Disentangled Representation Learning and Image Synthesis
• Selective Sampling and Mixture Models in Generative Adversarial Networks
• Unsupervised Meta-learning of Figure-Ground Segmentation via Imitating Visual Effects