Portrait Neural Radiance Fields from a Single Image
arXiv:2012.05903
Abstract
We present a method for estimating Neural Radiance Fields (NeRF) from a single headshot portrait. While NeRF has demonstrated high-quality view synthesis, it requires multiple images of static scenes and thus impractical for casual captures and moving subjects. In this work, we propose to pretrain the weights of a multilayer perceptron (MLP), which implicitly models the volumetric density and colors, with a meta-learning framework using a light stage portrait dataset. To improve the generalization to unseen faces, we train the MLP in the canonical coordinate space approximated by 3D face morphable models. We quantitatively evaluate the method using controlled captures and demonstrate the generalization to real portrait images, showing favorable results against state-of-the-arts.
Project webpage: https://portrait-nerf.github.io/
References in corpus (6)
- NeRF++: Analyzing and Improving Neural Radiance Fields
- DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
- Synthetic Depth-of-Field with a Single-Camera Mobile Phone
- First Order Motion Model for Image Animation
- Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
- Learning Compositional Radiance Fields of Dynamic Human Heads
Cited by in corpus (12)
- DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
- Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering
- ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields
- Neural Volume Rendering: NeRF And Beyond
- Neural Trajectory Fields for Dynamic Novel View Synthesis
- FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold
- AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
- NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo
- Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions
- Neural Fields in Visual Computing and Beyond
- DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing
- Learning to Stylize Novel Views