Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs

Abstract

Automatic synthesis of faces from visual attributes is an important problem in computer vision and has wide applications in law enforcement and entertainment. With the advent of deep generative convolutional neural networks (CNNs), attempts have been made to synthesize face images from attributes and text descriptions. In this paper, we take a different approach, where we formulate the original problem as a stage-wise learning problem. We first synthesize the facial sketch corresponding to the visual attributes and then we reconstruct the face image based on the synthesized sketch. The proposed Attribute2Sketch2Face framework, which is based on a combination of deep Conditional Variational Autoencoder (CVAE) and Generative Adversarial Networks (GANs), consists of three stages: (1) Synthesis of facial sketch from attributes using a CVAE architecture, (2) Enhancement of coarse sketches to produce sharper sketches using a GAN-based framework, and (3) Synthesis of face from sketch using another GAN-based network. Extensive experiments and comparison with recent methods are performed to verify the effectiveness of the proposed attribute-based three stage face synthesis method.

References in corpus (9)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Conditional Image Synthesis With Auxiliary Classifier GANs
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Unrolled Generative Adversarial Networks
• Mode Regularized Generative Adversarial Networks

Cited by in corpus (7)

• Polarimetric Thermal to Visible Face Verification via Attribute Preserved Synthesis
• Data-Driven Crowd Simulation with Generative Adversarial Networks
• Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs
• CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
• FTGAN: A Fully-trained Generative Adversarial Networks for Text to Face Generation
• High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks
• Attributes Aware Face Generation with Generative Adversarial Networks