118 citations · 275 across the 10 of their papers we have counts for
13 papers
No Distillation Needed: Single-Pass Real-Time Talking Heads via Acausal Noise Shaping
Yu Han, Dejan Markovic, Alexander Richard +4
Audio-driven facial animation underpins real-time avatars, telepresence, and embodied virtual agents. And it must run online: each frame emitted from audio observed up to the curre…
Neural Pixel Composition: 3D-4D View Synthesis from Multi-Views
Aayush Bansal, Michael Zollhoefer
We present Neural Pixel Composition (NPC), a novel approach for continuous 3D-4D view synthesis given only a discrete set of multi-view observations as input. Existing state-of-the…
iSDF: Real-Time Neural Signed Distance Fields for Robot Perception
Joseph Ortiz, Alexander Clegg, Jing Dong +4
We present iSDF, a continual learning system for real-time signed distance field (SDF) reconstruction. Given a stream of posed depth images from a moving camera, it trains a random…
KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints
Marko Mihajlovic, Aayush Bansal, Michael Zollhoefer +2
Image-based volumetric humans using pixel-aligned features promise generalization to unseen poses and identities. Prior work leverages global spatial encodings and multi-view geome…
COAP: Compositional Articulated Occupancy of People
Marko Mihajlovic, Shunsuke Saito, Aayush Bansal +2
We present a novel neural implicit representation for articulated human bodies. Compared to explicit template meshes, neural implicit body representations provide an efficient mech…
Mutual Scene Synthesis for Mixed Reality Telepresence
Mohammad Keshavarzi, Michael Zollhoefer, Allen Y. Yang +2
Remote telepresence via next-generation mixed reality platforms can provide higher levels of immersion for computer-mediated communications, allowing participants to engage in a wi…
A Deeper Look into DeepCap
Marc Habermann, Weipeng Xu, Michael Zollhoefer +2
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture…
Real-time Deep Dynamic Characters
Marc Habermann, Lingjie Liu, Weipeng Xu +3
We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view i…
Mixture of Volumetric Primitives for Efficient Neural Rendering
Stephen Lombardi, Tomas Simon, Gabriel Schwartz +3
Real-time rendering and animation of humans is a core function in games, movies, and telepresence applications. Existing methods have a number of drawbacks we aim to address with o…
A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose
Shih-Yang Su, Frank Yu, Michael Zollhoefer +1
While deep learning reshaped the classical motion capture pipeline with feed-forward networks, generative models are required to recover fine alignment via iterative refinement. Un…
PVA: Pixel-aligned Volumetric Avatars
Amit Raj, Michael Zollhoefer, Tomas Simon +4
Acquisition and rendering of photo-realistic human heads is a highly challenging research problem of particular importance for virtual telepresence. Currently, the highest quality…
DeepCap: Monocular Human Performance Capture Using Weak Supervision
Marc Habermann, Weipeng Xu, Michael Zollhoefer +2
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture…
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
Vincent Sitzmann, Michael Zollhöfer, Gordon Wetzstein
Unsupervised learning with generative models has the potential of discovering rich representations of 3D scenes. While geometric deep learning has explored 3D-structure-aware repre…