680 citations · 1.2k across the 34 of their papers we have counts for
36 papers
Latent Dynamics for Full Body Avatar Animation
Shichong Peng, Chengxiang Yin, Fei Jiang +7
Pose-driven full-body avatars built on neural rendering produce high-quality novel views of a captured subject. Yet loose clothing and other dynamic elements deform in ways pose al…
Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining
Junxuan Li, Rawal Khirodkar, Chengan He +37
High-quality 3D avatar modeling faces a critical trade-off between fidelity and generalization. On the one hand, multi-view studio data enables high-fidelity modeling of humans wit…
Audio Driven Real-Time Facial Animation for Social Telepresence
Jiye Lee, Chenghui Li, Linh Tran +5
We present an audio-driven real-time system for animating photorealistic 3D facial avatars with minimal latency, designed for social interactions in virtual reality for anyone. Cen…
Generative Modeling of Shape-Dependent Self-Contact Human Poses
Takehiko Ohkawa, Jihyun Lee, Shunsuke Saito +7
One can hardly model self-contact of human poses without considering underlying body shapes. For example, the pose of rubbing a belly for a person with a low BMI leads to penetrati…
HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars
Byungjun Kim, Shunsuke Saito, Giljoo Nam +4
We present a universal prior model for 3D head avatars with explicit hair compositionality. Existing approaches to build generalizable priors for 3D head avatars often adopt a holi…
Generative Head-Mounted Camera Captures for Photorealistic Avatars
Shaojie Bai, Seunghyeon Seo, Yida Wang +8
Enabling photorealistic avatar animations in virtual and augmented reality (VR/AR) has been challenging because of the difficulty of obtaining ground truth state of faces. It is ph…
REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioning
Jihyun Lee, Weipeng Xu, Alexander Richard +7
We present REWIND (Real-Time Egocentric Whole-Body Motion Diffusion), a one-step diffusion model for real-time, high-fidelity human motion estimation from egocentric image inputs.…
FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images
Rong Wang, Fabian Prada, Ziyan Wang +10
We present a novel method for reconstructing personalized 3D human avatars with realistic animation from only a few images. Due to the large variations in body shapes, poses, and c…
LUCAS: Layered Universal Codec Avatars
Di Liu, Teng Deng, Giljoo Nam +6
Photorealistic 3D head avatar reconstruction faces critical challenges in modeling dynamic face-hair interactions and achieving cross-identity generalization, particularly during e…
Relightable Full-Body Gaussian Codec Avatars
Shaofei Wang, Tomas Simon, Igor Santesteban +15
We propose Relightable Full-Body Gaussian Codec Avatars, a new approach for modeling relightable full-body avatars with fine-grained details including face and hands. The unique ch…
Universal Facial Encoding of Codec Avatars from VR Headsets
Shaojie Bai, Te-Li Wang, Chenghui Li +8
Faithful real-time facial animation is essential for avatar-mediated telepresence in Virtual Reality (VR). To emulate authentic communication, avatar animation needs to be efficien…
Rasterized Edge Gradients: Handling Discontinuities Differentiably
Stanislav Pidhorskyi, Tomas Simon, Gabriel Schwartz +3
Computing the gradients of a rendering process is paramount for diverse applications in computer vision and graphics. However, accurate computation of these gradients is challengin…
Fast Registration of Photorealistic Avatars for VR Facial Animation
Chaitanya Patel, Shaojie Bai, Te-Li Wang +2
Virtual Reality (VR) bares promise of social interactions that can feel more immersive than other media. Key to this is the ability to accurately animate a personalized photorealis…
A Local Appearance Model for Volumetric Capture of Diverse Hairstyle
Ziyan Wang, Giljoo Nam, Aljaz Bozic +4
Hair plays a significant role in personal identity and appearance, making it an essential component of high-quality, photorealistic avatars. Existing approaches either focus on mod…
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
Yonggan Fu, Yuecheng Li, Chenghui Li +4
Real-time and robust photorealistic avatars for telepresence in AR/VR have been highly desired for enabling immersive photorealistic telepresence. However, there still exists one k…
MEGANE: Morphable Eyeglass and Avatar Network
Junxuan Li, Shunsuke Saito, Tomas Simon +3
Eyeglasses play an important role in the perception of identity. Authentic virtual representations of faces can benefit greatly from their inclusion. However, modeling the geometri…
RelightableHands: Efficient Neural Relighting of Articulated Hand Models
Shun Iwase, Shunsuke Saito, Tomas Simon +7
We present the first neural relighting approach for rendering high-fidelity personalized hands that can be animated in real-time under novel illumination. Our approach adopts a tea…
NeuWigs: A Neural Dynamic Model for Volumetric Hair Capture and Animation
Ziyan Wang, Giljoo Nam, Tuur Stuyck +6
The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality. Both problems are highly challenging, because…
Drivable Volumetric Avatars using Texel-Aligned Features
Edoardo Remelli, Timur Bagautdinov, Shunsuke Saito +8
Photorealistic telepresence requires both high-fidelity body modeling and faithful driving to enable dynamically synthesized appearance that is indistinguishable from reality. In t…
Multiface: A Dataset for Neural Face Rendering
Cheng-hsin Wuu, Ningyuan Zheng, Scott Ardisson +28
Photorealistic avatars of human faces have come a long way in recent years, yet research along this area is limited by a lack of publicly available, high-quality datasets covering…
LiP-Flow: Learning Inference-time Priors for Codec Avatars via Normalizing Flows in Latent Space
Emre Aksan, Shugao Ma, Akin Caliskan +5
Neural face avatars that are trained from multi-view data captured in camera domes can produce photo-realistic 3D reconstructions. However, at inference time, they must be driven b…
Driving-Signal Aware Full-Body Avatars
Timur Bagautdinov, Chenglei Wu, Tomas Simon +6
We present a learning-based method for building driving-signal aware full-body avatars. Our model is a conditional variational autoencoder that can be animated with incomplete driv…
Pixel Codec Avatars
Shugao Ma, Tomas Simon, Jason Saragih +4
Telecommunication with photorealistic avatars in virtual or augmented reality is a promising path for achieving authentic face-to-face communication in 3D over remote physical dist…
Robust Egocentric Photo-realistic Facial Expression Transfer for Virtual Reality
Amin Jourabloo, Baris Gecer, Fernando De la Torre +7
Social presence, the feeling of being there with a real person, will fuel the next generation of communication systems driven by digital humans in virtual reality (VR). The best 3D…