Generating 3D faces using Convolutional Mesh Autoencoders

Abstract

Learned 3D representations of human faces are useful for computer vision problems such as 3D face tracking and reconstruction from images, as well as graphics applications such as character generation and animation. Traditional models learn a latent representation of a face using linear subspaces or higher-order tensor generalizations. Due to this linearity, they can not capture extreme deformations and non-linear expressions. To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface. We introduce mesh sampling operations that enable a hierarchical mesh representation that captures non-linear variations in shape and expression at multiple scales within the model. In a variational setting, our model samples diverse realistic 3D faces from a multivariate Gaussian distribution. Our training data consists of 20,466 meshes of extreme expressions captured over 12 different subjects. Despite limited training data, our trained model outperforms state-of-the-art face models with 50% lower reconstruction error, while using 75% fewer parameters. We also show that, replacing the expression space of an existing state-of-the-art face model with our autoencoder, achieves a lower reconstruction error. Our data, model and code are available at http://github.com/anuragranj/coma

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• Geometric deep learning: going beyond Euclidean data
• WaveNet: A Generative Model for Raw Audio
• Spectral Networks and Locally Connected Networks on Graphs
• Pixel Recurrent Neural Networks
• Deep Convolutional Networks on Graph-Structured Data
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
• Learning shape correspondence with anisotropic convolutional neural networks
• Review of Statistical Shape Spaces for 3D Data with Comparative Analysis for Human Faces
• Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
• Multilinear Wavelets: A Statistical Shape Space for Human Faces

Cited by in corpus (10)

• MeshCNN: A Network with an Edge
• Dynamic Graph CNN for Learning on Point Clouds
• MeshGAN: Non-linear 3D Morphable Models of Faces
• Single Image 3D Hand Reconstruction with Mesh Convolutions
• Neural Star Domain as Primitive Representation
• SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator
• Synthesizing facial photometries and corresponding geometries using generative adversarial networks
• A Decoupled 3D Facial Shape Model by Adversarial Training
• Geometrically Principled Connections in Graph Neural Networks
• High-Quality Face Capture Using Anatomical Muscles