145 citations · 447 across the 21 of their papers we have counts for
24 papers
Fast Image-based Neural Relighting with Translucency-Reflection Modeling
Shizhan Zhu, Shunsuke Saito, Aljaz Bozic +3
Image-based lighting (IBL) is a widely used technique that renders objects using a high dynamic range image or environment map. However, aggregating the irradiance at the object's…
NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objects
Taeksoo Kim, Shunsuke Saito, Hanbyul Joo
Deep generative models have been recently extended to synthesizing 3D digital humans. However, previous approaches treat clothed humans as a single chunk of geometry without consid…
MEGANE: Morphable Eyeglass and Avatar Network
Junxuan Li, Shunsuke Saito, Tomas Simon +3
Eyeglasses play an important role in the perception of identity. Authentic virtual representations of faces can benefit greatly from their inclusion. However, modeling the geometri…
RelightableHands: Efficient Neural Relighting of Articulated Hand Models
Shun Iwase, Shunsuke Saito, Tomas Simon +7
We present the first neural relighting approach for rendering high-fidelity personalized hands that can be animated in real-time under novel illumination. Our approach adopts a tea…
Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images
Radu Alexandru Rosu, Shunsuke Saito, Ziyan Wang +3
We present Neural Strands, a novel learning framework for modeling accurate hair geometry and appearance from multi-view image inputs. The learned hair model can be rendered in rea…
Drivable Volumetric Avatars using Texel-Aligned Features
Edoardo Remelli, Timur Bagautdinov, Shunsuke Saito +8
Photorealistic telepresence requires both high-fidelity body modeling and faithful driving to enable dynamically synthesized appearance that is indistinguishable from reality. In t…
Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing
Donglai Xiang, Timur Bagautdinov, Tuur Stuyck +10
Despite recent progress in developing animatable full-body avatars, realistic modeling of clothing - one of the core aspects of human self-expression - remains an open challenge. S…
KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints
Marko Mihajlovic, Aayush Bansal, Michael Zollhoefer +2
Image-based volumetric humans using pixel-aligned features promise generalization to unseen poses and identities. Prior work leverages global spatial encodings and multi-view geome…
COAP: Compositional Articulated Occupancy of People
Marko Mihajlovic, Shunsuke Saito, Aayush Bansal +2
We present a novel neural implicit representation for articulated human bodies. Compared to explicit template meshes, neural implicit body representations provide an efficient mech…
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
Ziqian Bai, Timur Bagautdinov, Javier Romero +3
Neural fields such as implicit surfaces have recently enabled avatar modeling from raw scans without explicit temporal correspondences. In this work, we exploit autoregressive mode…
Neural Fields in Visual Computing and Beyond
Yiheng Xie, Towaki Takikawa, Shunsuke Saito +7
Recent advances in machine learning have created increasing interest in solving visual computing problems using a class of coordinate-based neural networks that parametrize physica…
ARCH++: Animation-Ready Clothed Human Reconstruction Revisited
Tong He, Yuanlu Xu, Shunsuke Saito +2
We present ARCH++, an image-based method to reconstruct 3D avatars with arbitrary clothing styles. Our reconstructed avatars are animation-ready and highly realistic, in both the v…
SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements
Qianli Ma, Shunsuke Saito, Jinlong Yang +2
Learning to model and reconstruct humans in clothing is challenging due to articulation, non-rigid deformation, and varying clothing types and topologies. To enable learning, the c…
SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks
Shunsuke Saito, Jinlong Yang, Qianli Ma +1
We present SCANimate, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar. These avatars are driven by pose parame…
PVA: Pixel-aligned Volumetric Avatars
Amit Raj, Michael Zollhoefer, Tomas Simon +4
Acquisition and rendering of photo-realistic human heads is a highly challenging research problem of particular importance for virtual telepresence. Currently, the highest quality…
Monocular Real-Time Volumetric Performance Capture
Ruilong Li, Yuliang Xiu, Shunsuke Saito +3
We present the first approach to volumetric performance capture and novel-view rendering at real-time speed from monocular video, eliminating the need for expensive multi-view syst…
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Shunsuke Saito, Tomas Simon, Jason Saragih +1
Recent advances in image-based 3D human shape estimation have been driven by the significant improvement in representation power afforded by deep neural networks. Although current…
Learning to Infer Implicit Surfaces without 3D Supervision
Shichen Liu, Shunsuke Saito, Weikai Chen +1
Recent advances in 3D deep learning have shown that it is possible to train highly effective deep models for 3D shape generation, directly from 2D images. This is particularly inte…
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Shunsuke Saito, Zeng Huang, Ryota Natsume +3
We introduce Pixel-aligned Implicit Function (PIFu), a highly effective implicit representation that locally aligns pixels of 2D images with the global context of their correspondi…
SiCloPe: Silhouette-Based Clothed People
Ryota Natsume, Shunsuke Saito, Zeng Huang +4
We introduce a new silhouette-based representation for modeling clothed human bodies using deep generative models. Our method can reconstruct a complete and textured 3D model of a…
Learning Dense Facial Correspondences in Unconstrained Images
Ronald Yu, Shunsuke Saito, Haoxiang Li +2
We present a minimalistic but effective neural network that computes dense facial correspondences in highly unconstrained RGB images. Our network learns a per-pixel flow and a matc…
Photorealistic Facial Texture Inference Using Deep Neural Networks
Shunsuke Saito, Lingyu Wei, Liwen Hu +2
We present a data-driven inference method that can synthesize a photorealistic texture map of a complete 3D face model given a partial 2D view of a person in the wild. After an ini…
Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks
Samuli Laine, Tero Karras, Timo Aila +5
We present a real-time deep learning framework for video-based facial performance capture -- the dense 3D tracking of an actor's face given a monocular video. Our pipeline begins w…
Real-Time Facial Segmentation and Performance Capture from RGB Input
Shunsuke Saito, Tianye Li, Hao Li
We introduce the concept of unconstrained real-time 3D facial performance capture through explicit semantic segmentation in the RGB input. To ensure robustness, cutting edge superv…