Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model

Abstract

We propose a novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with wide ranges of expressions, poses, and illuminations conditioned by a 3D morphable model. Previous adversarial style-transfer methods either supervise their networks with large volume of paired data or use unpaired data with a highly under-constrained two-way generative framework in an unsupervised fashion. We introduce pairwise adversarial supervision to constrain two-way domain adaptation by a small number of paired real and synthetic images for training along with the large volume of unpaired data. Extensive qualitative and quantitative experiments are performed to validate our idea. Generated face images of new identities contain pose, lighting and expression diversity and qualitative results show that they are highly constraint by the synthetic input image while adding photorealism and retaining identity information. We combine face images generated by the proposed method with the real data set to train face recognition algorithms. We evaluated the model on two challenging data sets: LFW and IJB-A. We observe that the generated images from our framework consistently improves over the performance of deep face recognition network trained with Oxford VGG Face dataset and achieves comparable results to the state-of-the-art.

References in corpus (5)

• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
• Learning from Synthetic Humans
• Appearance-Based Gaze Estimation in the Wild
• Learning Deep Convolutional Embeddings for Face Representation Using Joint Sample- and Set-based Supervision

Cited by in corpus (7)

• Fast Parallel Hypertree Decompositions in Logarithmic Recursion Depth
• GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction
• A Survey on Face Data Augmentation
• Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networks
• Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction
• OSTeC: One-Shot Texture Completion
• Sampling Strategies for GAN Synthetic Data