DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)
arXiv:1802.06454
Abstract
Unsupervised image translation, which aims in translating two independent sets of images, is challenging in discovering the correct correspondences without paired data. Existing works build upon Generative Adversarial Network (GAN) such that the distribution of the translated images are indistinguishable from the distribution of the target set. However, such set-level constraints cannot learn the instance-level correspondences (e.g. aligned semantic parts in object configuration task). This limitation often results in false positives (e.g. geometric or semantic artifacts), and further leads to mode collapse problem. To address the above issues, we propose a novel framework for instance-level image translation by Deep Attention GAN (DA-GAN). Such a design enables DA-GAN to decompose the task of translating samples from two sets into translating instances in a highly-structured latent space. Specifically, we jointly learn a deep attention encoder, and the instancelevel correspondences could be consequently discovered through attending on the learned instance pairs. Therefore, the constraints could be exploited on both set-level and instance-level. Comparisons against several state-ofthe- arts demonstrate the superiority of our approach, and the broad application capability, e.g, pose morphing, data augmentation, etc., pushes the margin of domain translation problem.
arXiv commentComputer Vision and Pattern Recognition
References in corpus (16)
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
- Deep Residual Learning for Image Recognition
- Generative Adversarial Text to Image Synthesis
- Conditional Image Synthesis With Auxiliary Classifier GANs
- Conditional Image Generation with PixelCNN Decoders
- Improved Techniques for Training GANs
- InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- Energy-based Generative Adversarial Network
- Texture Networks: Feed-forward Synthesis of Textures and Stylized Images
- Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
- Unrolled Generative Adversarial Networks
- Amortised MAP Inference for Image Super-resolution
- Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts
- Attribute2Image: Conditional Image Generation from Visual Attributes
- Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
- Scribbler: Controlling Deep Image Synthesis with Sketch and Color