Synthesizing facial photometries and corresponding geometries using generative adversarial networks

Abstract

Artificial data synthesis is currently a well studied topic with useful applications in data science, computer vision, graphics and many other fields. Generating realistic data is especially challenging since human perception is highly sensitive to non realistic appearance. In recent times, new levels of realism have been achieved by advances in GAN training procedures and architectures. These successful models, however, are tuned mostly for use with regularly sampled data such as images, audio and video. Despite the successful application of the architecture on these types of media, applying the same tools to geometric data poses a far greater challenge. The study of geometric deep learning is still a debated issue within the academic community as the lack of intrinsic parametrization inherent to geometric objects prohibits the direct use of convolutional filters, a main building block of today's machine learning systems. In this paper we propose a new method for generating realistic human facial geometries coupled with overlayed textures. We circumvent the parametrization issue by imposing a global mapping from our data to the unit rectangle. We further discuss how to design such a mapping to control the mapping distortion and conserve area within the mapped image. By representing geometric textures and geometries as images, we are able to use advanced GAN methodologies to generate new geometries. We address the often neglected topic of relation between texture and geometry and propose to use this correlation to match between generated textures and their corresponding geometries. We offer a new method for training GAN models on partially corrupted data. Finally, we provide empirical evidence demonstrating our generative model's ability to produce examples of new identities independent from the training data while maintaining a high level of realism, two traits that are often at odds.

arXiv comment23 pages, 16 figures, 3 tables

References in corpus (8)

• Geometric deep learning: going beyond Euclidean data
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Progressive Growing of GANs for Improved Quality, Stability, and Variation
• Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis
• 3D Morphable Models as Spatial Transformer Networks
• Generating 3D faces using Convolutional Mesh Autoencoders
• Nonlinear 3D Face Morphable Model
• High Quality Facial Surface and Texture Synthesis via Generative Adversarial Networks

Cited by in corpus (2)

• Learning Formation of Physically-Based Face Attributes
• A Decoupled 3D Facial Shape Model by Adversarial Training