112 citations · 632 across the 28 of their papers we have counts for
34 papers
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
Ziqian Bai, Timur Bagautdinov, Javier Romero +3
Neural fields such as implicit surfaces have recently enabled avatar modeling from raw scans without explicit temporal correspondences. In this work, we exploit autoregressive mode…
Advances in Neural Rendering
Ayush Tewari, Justus Thies, Ben Mildenhall +14
Synthesizing photo-realistic images and videos is at the heart of computer graphics and has been the focus of decades of research. Traditionally, synthetic images of a scene are ge…
NRST: Non-rigid Surface Tracking from Monocular Video
Marc Habermann, Weipeng Xu, Helge Rhodin +3
We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-ri…
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement
Alexander Richard, Michael Zollhoefer, Yandong Wen +2
This paper presents a generic method for generating full facial 3D animation from speech. Existing approaches to audio-driven facial animation exhibit uncanny or static upper face…
Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video
Edgar Tretschk, Ayush Tewari, Vladislav Golyanik +3
We present Non-Rigid Neural Radiance Fields (NR-NeRF), a reconstruction and novel view synthesis approach for general non-rigid dynamic scenes. Our approach takes RGB images of a d…
Face2Face: Real-time Face Capture and Reenactment of RGB Videos
Justus Thies, Michael Zollhöfer, Marc Stamminger +2
We present Face2Face, a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video st…
StyleRig: Rigging StyleGAN for 3D Control over Portrait Images
Ayush Tewari, Mohamed Elgharib, Gaurav Bharaj +5
StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face paramet…
Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation
Lingjie Liu, Weipeng Xu, Marc Habermann +5
Synthesizing realistic videos of humans using neural networks has been a popular alternative to the conventional graphics-based rendering pipeline due to its high efficiency. Exist…
DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data
Aljaž Božič, Michael Zollhöfer, Christian Theobalt +1
Applying data-driven approaches to non-rigid 3D reconstruction has been difficult, which we believe can be attributed to the lack of a large-scale training corpus. Unfortunately, t…
Neural Style-Preserving Visual Dubbing
Hyeongwoo Kim, Mohamed Elgharib, Michael Zollhöfer +4
Dubbing is a technique for translating video content from one language to another. However, state-of-the-art visual dubbing techniques directly copy facial expressions from source…
3D Morphable Face Models -- Past, Present and Future
Bernhard Egger, William A. P. Smith, Ayush Tewari +10
In this paper, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed. The challenges in building and applying these models, name…
Real-Time Global Illumination Decomposition of Videos
Abhimitra Meka, Mohammad Shafiei, Michael Zollhoefer +2
We propose the first approach for the decomposition of a monocular color video into direct and indirect illumination components in real time. We retrieve, in separate layers, the c…
Text-based Editing of Talking-head Video
Ohad Fried, Ayush Tewari, Michael Zollhöfer +7
Editing talking-head video to change the speech content or to remove filler words is challenging. We propose a novel method to edit talking-head video based on its transcript to pr…
DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects
Edgar Tretschk, Ayush Tewari, Michael Zollhöfer +2
Mesh autoencoders are commonly used for dimensionality reduction, sampling and mesh modeling. We propose a general-purpose DEep MEsh Autoencoder (DEMEA) which adds a novel embedded…
Deferred Neural Rendering: Image Synthesis using Neural Textures
Justus Thies, Michael Zollhöfer, Matthias Nießner
The modern computer graphics pipeline can synthesize images at remarkable visual quality; however, it requires well-defined, high-quality 3D content as input. In this work, we expl…
FML: Face Model Learning from Videos
Ayush Tewari, Florian Bernard, Pablo Garrido +6
Monocular image-based 3D reconstruction of faces is a long-standing problem in computer vision. Since image data is a 2D projection of a 3D face, the resulting depth ambiguity make…
DeepVoxels: Learning Persistent 3D Feature Embeddings
Vincent Sitzmann, Justus Thies, Felix Heide +3
In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis. To this end, we propose Dee…
IGNOR: Image-guided Neural Object Rendering
Justus Thies, Michael Zollhöfer, Christian Theobalt +2
We propose a learned image-guided rendering technique that combines the benefits of image-based rendering and GAN-based image synthesis. The goal of our method is to generate photo…
LiveCap: Real-time Human Performance Capture from Monocular Video
Marc Habermann, Weipeng Xu, Michael Zollhoefer +2
We present the first real-time human performance capture approach that reconstructs dense, space-time coherent deforming geometry of entire humans in general everyday clothing from…
Neural Rendering and Reenactment of Human Actor Videos
Lingjie Liu, Weipeng Xu, Michael Zollhoefer +5
We propose a method for generating video-realistic animations of real humans under user control. In contrast to conventional human character rendering, we do not require the availa…
HeadOn: Real-time Reenactment of Human Portrait Videos
Justus Thies, Michael Zollhöfer, Christian Theobalt +2
We propose HeadOn, the first real-time source-to-target reenactment approach for complete human portrait videos that enables transfer of torso and head motion, face expression, and…
Deep Video Portraits
Hyeongwoo Kim, Pablo Garrido, Ayush Tewari +7
We present a novel approach that enables photo-realistic re-animation of portrait videos using only an input video. In contrast to existing approaches that are restricted to manipu…
Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera
Weipeng Xu, Avishek Chatterjee, Michael Zollhoefer +4
We propose the first real-time approach for the egocentric estimation of 3D human body pose in a wide range of unconstrained everyday activities. This setting has a unique set of c…
LIME: Live Intrinsic Material Estimation
Abhimitra Meka, Maxim Maximov, Michael Zollhoefer +4
We present the first end to end approach for real time material estimation for general object shapes with uniform material that only requires a single color image as input. In addi…