CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training

Abstract

We present variational generative adversarial networks, a general learning framework that combines a variational auto-encoder with a generative adversarial network, for synthesizing images in fine-grained categories, such as faces of a specific person or objects in a category. Our approach models an image as a composition of label and latent attributes in a probabilistic model. By varying the fine-grained category label fed into the resulting generative model, we can generate images in a specific category with randomly drawn values on a latent attribute vector. Our approach has two novel aspects. First, we adopt a cross entropy loss for the discriminative and classifier network, but a mean discrepancy objective for the generative network. This kind of asymmetric loss function makes the GAN training more stable. Second, we adopt an encoder network to learn the relationship between the latent space and the real image space, and use pairwise feature matching to keep the structure of generated images. We experiment with natural images of faces, flowers, and birds, and demonstrate that the proposed models are capable of generating realistic and diverse samples with fine-grained category labels. We further show that our models can be applied to other tasks, such as image inpainting, super-resolution, and data augmentation for training better face recognition models.

to appear in ICCV 2017

References in corpus (6)

• Conditional Generative Adversarial Nets
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Learning Face Representation from Scratch
• Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space
• Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
• McGan: Mean and Covariance Feature Matching GAN

Cited by in corpus (16)

• Improving Outfit Recommendation with Co-supervision of Fashion Generation
• Improving Generalization for Abstract Reasoning Tasks Using Disentangled Feature Representations
• Towards Open-Set Identity Preserving Face Synthesis
• Improved Training of Generative Adversarial Networks Using Representative Features
• Structured Variational Inference for Simulating Populations of Radio Galaxies
• 3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations
• Lung CT Imaging Sign Classification through Deep Learning on Small Data
• Variational Capsules for Image Analysis and Synthesis
• Pay attention! - Robustifying a Deep Visuomotor Policy through Task-Focused Attention
• RNN-based Generative Model for Fine-Grained Sketching
• Student's t-Generative Adversarial Networks
• Fast Face Image Synthesis with Minimal Training
• Latent Transformations for Object View Points Synthesis
• Synthesizing Photorealistic Images with Deep Generative Learning
• Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation
• Capturing Evolution Genes for Time Series Data