Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation

Abstract

In this work, we propose a novel Cycle In Cycle Generative Adversarial Network (C${}^2$GAN) for the task of keypoint-guided image generation. The proposed C${}^2$GAN is a cross-modal framework exploring a joint exploitation of the keypoint and the image data in an interactive manner. C${}^2$GAN contains two different types of generators, i.e., keypoint-oriented generator and image-oriented generator. Both of them are mutually connected in an end-to-end learnable fashion and explicitly form three cycled sub-networks, i.e., one image generation cycle and two keypoint generation cycles. Each cycle not only aims at reconstructing the input domain, and also produces useful output involving in the generation of another cycle. By so doing, the cycles constrain each other implicitly, which provides complementary information from the two different modalities and brings extra supervision across cycles, thus facilitating more robust optimization of the whole network. Extensive experimental results on two publicly available datasets, i.e., Radboud Faces and Market-1501, demonstrate that our approach is effective to generate more photo-realistic images compared with state-of-the-art models.

arXiv comment9 pages, 8 figures, accepted to ACM MM 2019

References in corpus (18)

• Conditional Generative Adversarial Nets
• WaveNet: A Generative Model for Raw Audio
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Improved Training of Wasserstein GANs
• Improved Techniques for Training GANs
• Pose Guided Person Image Generation
• Invertible Conditional GANs for image editing
• Semi-Supervised Learning with Generative Adversarial Networks
• DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
• Unsupervised Attention-guided Image to Image Translation
• Semantic Image Synthesis with Spatially-Adaptive Normalization
• InstaGAN: Instance-aware Image-to-Image Translation
• GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data
• Every Smile is Unique: Landmark-Guided Diverse Smile Generation
• GazeCorrection:Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks
• Cascade Attention Guided Residue Learning GAN for Cross-Modal Translation

Cited by in corpus (1)

• Total Generate: Cycle in Cycle Generative Adversarial Networks for Generating Human Faces, Hands, Bodies, and Natural Scenes