38 citations · 156 across the 18 of their papers we have counts for
13 papers · 1 filter
Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions
Rawal Khirodkar, Jyun-Ting Song, Jinkun Cao +2
Understanding how humans interact with each other is key to building realistic multi-human virtual reality systems. This area remains relatively unexplored due to the lack of large…
Real-Time Simulated Avatar from Head-Mounted Sensors
Zhengyi Luo, Jinkun Cao, Rawal Khirodkar +4
We present SimXR, a method for controlling a simulated avatar from information (headset pose and cameras) obtained from AR / VR headsets. Due to the challenging viewpoint of head-m…
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani +98
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric…
Universal Humanoid Motion Representations for Physics-Based Control
Zhengyi Luo, Jinkun Cao, Josh Merel +4
We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. Due to the high dimensionality of humanoids…
Perpetual Humanoid Control for Real-time Simulated Avatars
Zhengyi Luo, Jinkun Cao, Alexander Winkler +2
We present a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior in the presence of noisy input (e.g. pose estimates from vid…
Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
Davis Rempe, Zhengyi Luo, Xue Bin Peng +5
We introduce a method for generating realistic pedestrian trajectories and full-body animations that can be controlled to meet user-defined goals. We draw on recent advances in gui…
Embodied Scene-aware Human Pose Estimation
Zhengyi Luo, Shun Iwase, Ye Yuan +1
We propose embodied scene-aware human pose estimation where we estimate 3D poses based on a simulated agent's proprioception and scene awareness, along with external third-person o…
Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
Zhengyi Luo, Ryo Hachiuma, Ye Yuan +1
We propose a method for object-aware 3D egocentric pose estimation that tightly integrates kinematics modeling, dynamics modeling, and scene object information. Unlike prior kinema…
Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation
Zhengyi Luo, Ryo Hachiuma, Ye Yuan +2
We propose a method for incorporating object interaction and human body dynamics into the task of 3D ego-pose estimation using a head-mounted camera. We use a kinematics model of t…
3D Human Motion Estimation via Motion Compression and Refinement
Zhengyi Luo, S. Alireza Golestaneh, Kris M. Kitani
We develop a technique for generating smooth and accurate 3D human pose and motion estimates from RGB video sequences. Our method, which we call Motion Estimation via Variational A…
Learning Shape Representations for Clothing Variations in Person Re-Identification
Yu-Jhe Li, Zhengyi Luo, Xinshuo Weng +1
Person re-identification (re-ID) aims to recognize instances of the same person contained in multiple images taken across different cameras. Existing methods for re-ID tend to rely…
Cross-Domain 3D Equivariant Image Embeddings
Carlos Esteves, Avneesh Sud, Zhengyi Luo +2
Spherical convolutional networks have been introduced recently as tools to learn powerful feature representations of 3D shapes. Spherical CNNs are equivariant to 3D rotations makin…
Cloud Chaser: Real Time Deep Learning Computer Vision on Low Computing Power Devices
Zhengyi Luo, Austin Small, Liam Dugan +1
Internet of Things(IoT) devices, mobile phones, and robotic systems are often denied the power of deep learning algorithms due to their limited computing power. However, to provide…