Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networks

Abstract

Generating realistic 3D faces is of high importance for computer graphics and computer vision applications. Generally, research on 3D face generation revolves around linear statistical models of the facial surface. Nevertheless, these models cannot represent faithfully either the facial texture or the normals of the face, which are very crucial for photo-realistic face synthesis. Recently, it was demonstrated that Generative Adversarial Networks (GANs) can be used for generating high-quality textures of faces. Nevertheless, the generation process either omits the geometry and normals, or independent processes are used to produce 3D shape information. In this paper, we present the first methodology that generates high-quality texture, shape, and normals jointly, which can be used for photo-realistic synthesis. To do so, we propose a novel GAN that can generate data from different modalities while exploiting their correlations. Furthermore, we demonstrate how we can condition the generation on the expression and create faces with various facial expressions. The qualitative results shown in this paper are compressed due to size limitations, full-resolution results and the accompanying video can be found in the supplementary documents. The code and models are available at the project page: https://github.com/barisgecer/TBGAN.

arXiv commentCheck project page: https://github.com/barisgecer/TBGAN for the full resolution results and the accompanying video

References in corpus (5)

• GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction
• Representation Learning by Rotating Your Faces
• Review of Statistical Shape Spaces for 3D Data with Comparative Analysis for Human Faces
• Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model
• Learning Deep Convolutional Embeddings for Face Representation Using Joint Sample- and Set-based Supervision

Cited by in corpus (6)

• Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs
• OSTeC: One-Shot Texture Completion
• 3D Generative Model Latent Disentanglement via Local Eigenprojection
• REFA: Real-time Egocentric Facial Animations for Virtual Reality
• Inverting Generative Adversarial Renderer for Face Reconstruction