Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions

Abstract

We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation $\unicode{x2013}$ neural implicit functions $\unicode{x2013}$ which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)
• NeRF++: Analyzing and Improving Neural Radiance Fields
• GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction
• StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
• 3D Face Reconstruction with Geometry Details from a Single Image
• MetaSDF: Meta-learning Signed Distance Functions
• Unsupervised Generative 3D Shape Learning from Natural Images
• Portrait Neural Radiance Fields from a Single Image