81 citations · 171 across the 29 of their papers we have counts for
32 papers
Archon: A Unified Multimodal Model for Holistic Digital Human Generation
Chong Bao, Shichen Liu, Lijun Yu +9
Digital humans are fundamental to immersive interaction, yet creating a unified model for holistic modalities, including text, audio, motion, and visual content, remains an open ch…
Talking Together: Synthesizing Co-Located 3D Conversations from Audio
Mengyi Shan, Shouchieh Chang, Ziqian Bai +6
We tackle the challenging task of generating complete 3D facial animations for two interacting, co-located participants from a mixed audio stream. While existing methods often prod…
S^2VG: 3D Stereoscopic and Spatial Video Generation via Denoising Frame Matrix
Peng Dai, Feitong Tan, Qiangeng Xu +6
While video generation models excel at producing high-quality monocular videos, generating 3D stereoscopic and spatial videos for immersive applications remains an underexplored ch…
IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos
Yuan Li, Ziqian Bai, Feitong Tan +3
We propose a novel 3D-aware diffusion-based method for generating photorealistic talking head videos directly from a single identity image and explicit control signals (e.g., expre…
LightAvatar: Efficient Head Avatar as Dynamic Neural Light Field
Huan Wang, Feitong Tan, Ziqian Bai +9
Recent works have shown that neural radiance fields (NeRFs) on top of parametric models have reached SOTA quality to build photorealistic head avatars from a monocular video. Howev…
SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
Peng Dai, Feitong Tan, Qiangeng Xu +5
Video generation models have demonstrated great capabilities of producing impressive monocular videos, however, the generation of 3D stereoscopic video remains under-explored. We p…
Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes
Ziqian Bai, Feitong Tan, Sean Fanello +5
3D head avatars built with neural implicit volumetric representations have achieved unprecedented levels of photorealism. However, the computational cost of these methods remains a…
CHOSEN: Contrastive Hypothesis Selection for Multi-View Depth Refinement
Di Qiu, Yinda Zhang, Thabo Beeler +5
We propose CHOSEN, a simple yet flexible, robust and effective multi-view depth refinement framework. It can be employed in any existing multi-view stereo pipeline, with straightfo…
One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptation
Zhixuan Yu, Ziqian Bai, Abhimitra Meka +6
Traditional methods for constructing high-quality, personalized head avatars from monocular videos demand extensive face captures and training time, posing a significant challenge…
Sandwiched Compression: Repurposing Standard Codecs with Neural Network Wrappers
Onur G. Guleryuz, Philip A. Chou, Berivan Isik +6
We propose sandwiching standard image and video codecs between pre- and post-processing neural networks. The networks are jointly trained through a differentiable codec proxy to mi…
MVDD: Multi-View Depth Diffusion Models
Zhen Wang, Qiangeng Xu, Feitong Tan +6
Denoising diffusion models have demonstrated outstanding results in 2D image generation, yet it remains a challenge to replicate its success in 3D shape generation. In this paper,…
Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing
Yushi Lan, Feitong Tan, Di Qiu +8
We present a novel framework for generating photorealistic 3D human head and subsequently manipulating and reposing them with remarkable flexibility. The proposed approach leverage…
Controllable Light Diffusion for Portraits
David Futschik, Kelvin Ritland, James Vecore +5
We introduce light diffusion, a novel method to improve lighting in portraits, softening harsh shadows and specular highlights while preserving overall scene illumination. Inspired…
Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videos
Ziqian Bai, Feitong Tan, Zeng Huang +12
We propose a method to learn a high-quality implicit 3D head avatar from a monocular RGB video captured in the wild. The learnt avatar is driven by a parametric face model to achie…
Multi-Camera Lighting Estimation for Photorealistic Front-Facing Mobile Augmented Reality
Yiqin Zhao, Sean Fanello, Tian Guo
Lighting understanding plays an important role in virtual object composition, including mobile augmented reality (AR) applications. Prior work often targets recovering lighting fro…
Neural Rendering in a Room: Amodal 3D Understanding and Free-Viewpoint Rendering for the Closed Scene Composed of Pre-Captured Objects
Bangbang Yang, Yinda Zhang, Yijin Li +4
We, as human beings, can understand and picture a familiar scene from arbitrary viewpoints given a single image, whereas this is still a grand challenge for computers. We hereby pr…
VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting
Feitong Tan, Sean Fanello, Abhimitra Meka +6
We propose VoLux-GAN, a generative framework to synthesize 3D-aware faces with convincing relighting. Our main contribution is a volumetric HDRI relighting method that can efficien…
Advances in Neural Rendering
Ayush Tewari, Justus Thies, Ben Mildenhall +14
Synthesizing photo-realistic images and videos is at the heart of computer graphics and has been the focus of decades of research. Traditionally, synthetic images of a scene are ge…
Multiresolution Deep Implicit Functions for 3D Shape Representation
Zhang Chen, Yinda Zhang, Kyle Genova +7
We introduce Multiresolution Deep Implicit Functions (MDIF), a hierarchical representation that can recover fine geometry detail, while being able to perform global operations such…
HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences
Feitong Tan, Danhang Tang, Mingsong Dou +9
In this paper, we address the problem of building dense correspondences between human images under arbitrary camera viewpoints and body poses. Prior art either assumes small motion…
Light Stage Super-Resolution: Continuous High-Frequency Relighting
Tiancheng Sun, Zexiang Xu, Xiuming Zhang +6
The light stage has been widely used in computer graphics for the past two decades, primarily to enable the relighting of human faces. By capturing the appearance of the human subj…
Learning Illumination from Diverse Portraits
Chloe LeGendre, Wan-Chun Ma, Rohit Pandey +5
We present a learning-based technique for estimating high dynamic range (HDR), omnidirectional illumination from a single low dynamic range (LDR) portrait image captured under arbi…
Neural Light Transport for Relighting and View Synthesis
Xiuming Zhang, Sean Fanello, Yun-Ta Tsai +10
The light transport (LT) of a scene describes how it appears under different lighting and viewing directions, and complete knowledge of a scene's LT enables the synthesis of novel…
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
Vladimir Tankovich, Christian Häne, Yinda Zhang +3
This paper presents HITNet, a novel neural network architecture for real-time stereo matching. Contrary to many recent neural network approaches that operate on a full cost volume…