1.2k citations · 2k across the 39 of their papers we have counts for
41 papers
REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioning
Jihyun Lee, Weipeng Xu, Alexander Richard +7
We present REWIND (Real-Time Egocentric Whole-Body Motion Diffusion), a one-step diffusion model for real-time, high-fidelity human motion estimation from egocentric image inputs.…
Omnigrasp: Grasping Diverse Objects with Simulated Humanoids
Zhengyi Luo, Jinkun Cao, Sammy Christen +3
We present a method for controlling a simulated humanoid to grasp an object and move it to follow an object's trajectory. Due to the challenges in controlling a humanoid with dexte…
Authentic Hand Avatar from a Phone Scan via Universal Hand Model
Gyeongsik Moon, Weipeng Xu, Rohan Joshi +2
The authentic 3D hand avatar with every identifiable information, such as hand shapes and textures, is necessary for immersive experiences in AR/VR. In this paper, we present a uni…
Real-Time Simulated Avatar from Head-Mounted Sensors
Zhengyi Luo, Jinkun Cao, Rawal Khirodkar +4
We present SimXR, a method for controlling a simulated avatar from information (headset pose and cameras) obtained from AR / VR headsets. Due to the challenging viewpoint of head-m…
A Dataset of Relighted 3D Interacting Hands
Gyeongsik Moon, Shunsuke Saito, Weipeng Xu +12
The two-hand interaction is one of the most challenging signals to analyze due to the self-similarity, complicated articulations, and occlusions of hands. Although several datasets…
Universal Humanoid Motion Representations for Physics-Based Control
Zhengyi Luo, Jinkun Cao, Josh Merel +4
We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. Due to the high dimensionality of humanoids…
Perpetual Humanoid Control for Real-time Simulated Avatars
Zhengyi Luo, Jinkun Cao, Alexander Winkler +2
We present a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior in the presence of noisy input (e.g. pose estimates from vid…
Scene-aware Egocentric 3D Human Pose Estimation
Jian Wang, Lingjie Liu, Weipeng Xu +3
Egocentric 3D human pose estimation with a single head-mounted fisheye camera has recently attracted attention due to its numerous applications in virtual and augmented reality. Ex…
HDHumans: A Hybrid Approach for High-fidelity Digital Humans
Marc Habermann, Lingjie Liu, Weipeng Xu +3
Photo-real digital human avatars are of enormous importance in graphics, as they enable immersive communication over the globe, improve gaming and entertainment experiences, and ca…
Dressing Avatars: Deep Photorealistic Appearance for Physically Simulated Clothing
Donglai Xiang, Timur Bagautdinov, Tuur Stuyck +10
Despite recent progress in developing animatable full-body avatars, realistic modeling of clothing - one of the core aspects of human self-expression - remains an open challenge. S…
HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance
Soshi Shimada, Vladislav Golyanik, Zhi Li +3
Marker-less monocular 3D human motion capture (MoCap) with scene interactions is a challenging research topic relevant for extended reality, robotics and virtual avatar generation.…
Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision
Jian Wang, Lingjie Liu, Weipeng Xu +3
Egocentric 3D human pose estimation with a single fisheye camera has drawn a significant amount of attention recently. However, existing methods struggle with pose estimation from…
A Deeper Look into DeepCap
Marc Habermann, Weipeng Xu, Michael Zollhoefer +2
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture…
NRST: Non-rigid Surface Tracking from Monocular Video
Marc Habermann, Weipeng Xu, Helge Rhodin +3
We propose an efficient method for non-rigid surface tracking from monocular RGB videos. Given a video and a template mesh, our algorithm sequentially registers the template non-ri…
Driving-Signal Aware Full-Body Avatars
Timur Bagautdinov, Chenglei Wu, Tomas Simon +6
We present a learning-based method for building driving-signal aware full-body avatars. Our model is a conditional variational autoencoder that can be animated with incomplete driv…
Modeling Clothing as a Separate Layer for an Animatable Human Avatar
Donglai Xiang, Fabian Prada, Timur Bagautdinov +5
We have recently seen great progress in building photorealistic animatable full-body codec avatars, but generating high-fidelity animation of clothing is still difficult. To addres…
Neural Monocular 3D Human Motion Capture with Physical Awareness
Soshi Shimada, Vladislav Golyanik, Weipeng Xu +2
We present a new trainable system for physically plausible markerless 3D human motion capture, which achieves state-of-the-art results in a broad range of challenging scenarios. Un…
Revitalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation
Taosha Fan, Kalyan Vasudev Alwala, Donglai Xiang +3
We propose a novel sparse constrained formulation and from it derive a real-time optimization method for 3D human pose and shape estimation. Our optimization method, SCOPE (Sparse…
Real-time Deep Dynamic Characters
Marc Habermann, Lingjie Liu, Weipeng Xu +3
We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance learned in a new weakly supervised way from multi-view i…
Estimating Egocentric 3D Human Pose in Global Space
Jian Wang, Lingjie Liu, Weipeng Xu +2
Egocentric 3D human pose estimation using a single fisheye camera has become popular recently as it allows capturing a wide range of daily activities in unconstrained environments,…
Learning Speech-driven 3D Conversational Gestures from Video
Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta +5
We propose the first approach to automatically and jointly synthesize both the synchronous 3D conversational body and hand gestures, as well as 3D face and head animations, of a vi…
Neural Re-Rendering of Humans from a Single Image
Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu +2
Human re-rendering from a single image is a starkly under-constrained problem, and state-of-the-art algorithms often exhibit undesired artefacts, such as over-smoothing, unrealisti…
PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time
Soshi Shimada, Vladislav Golyanik, Weipeng Xu +1
Marker-less 3D human motion capture from a single colour camera has seen significant progress. However, it is a very challenging and severely ill-posed problem. In consequence, eve…
DeepCap: Monocular Human Performance Capture Using Weak Supervision
Marc Habermann, Weipeng Xu, Michael Zollhoefer +2
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture…