20 citations · 34 across the 16 of their papers we have counts for
16 papers
Ground Reaction Inertial Poser: Physics-based Human Motion Capture from Sparse IMUs and Insole Pressure Sensors
Ryosuke Hori, Jyun-Ting Song, Zhengyi Luo +4
We propose Ground Reaction Inertial Poser (GRIP), a method that reconstructs physically plausible human motion using four wearable devices. Unlike conventional IMU-only approaches,…
BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
Yitang Li, Zhengyi Luo, Tonghe Zhang +10
Building Behavioral Foundation Models (BFMs) for humanoid robots has the potential to unify diverse control tasks under a single, promptable generalist policy. However, existing ap…
Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning
Zhengyi Luo, Chen Tessler, Toru Lin +8
Human behavior is fundamentally shaped by visual perception -- our ability to interact with the world depends on actively gathering relevant information and adapting our movements…
CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow
Takahiro Maeda, Jinkun Cao, Norimichi Ukita +1
Many density estimation techniques for 3D human motion prediction require a significant amount of inference time, often exceeding the duration of the predicted time horizon. To add…
ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills
Tairan He, Jiawei Gao, Wenli Xiao +15
Humanoid robots hold the potential for unparalleled versatility in performing human-like, whole-body skills. However, achieving agile and coordinated whole-body motions remains a s…
Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions
Rawal Khirodkar, Jyun-Ting Song, Jinkun Cao +2
Understanding how humans interact with each other is key to building realistic multi-human virtual reality systems. This area remains relatively unexplored due to the lack of large…
Joint Diffusion for Universal Hand-Object Grasp Generation
Jinkun Cao, Jingyuan Liu, Kris Kitani +1
Predicting and generating human hand grasp over objects is critical for animation and robotic tasks. In this work, we focus on generating both the hand and objects in a grasp by a…
Omnigrasp: Grasping Diverse Objects with Simulated Humanoids
Zhengyi Luo, Jinkun Cao, Sammy Christen +3
We present a method for controlling a simulated humanoid to grasp an object and move it to follow an object's trajectory. Due to the challenges in controlling a humanoid with dexte…
SMPLOlympics: Sports Environments for Physically Simulated Humanoids
Zhengyi Luo, Jiashun Wang, Kangni Liu +9
We present SMPLOlympics, a collection of physically simulated environments that allow humanoids to compete in a variety of Olympic sports. Sports simulation offers a rich and stand…
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
Tairan He, Zhengyi Luo, Xialin He +6
We present OmniH2O (Omni Human-to-Humanoid), a learning-based system for whole-body humanoid teleoperation and autonomy. Using kinematic pose as a universal control interface, Omni…
Real-Time Simulated Avatar from Head-Mounted Sensors
Zhengyi Luo, Jinkun Cao, Rawal Khirodkar +4
We present SimXR, a method for controlling a simulated avatar from information (headset pose and cameras) obtained from AR / VR headsets. Due to the challenging viewpoint of head-m…
Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
Tairan He, Zhengyi Luo, Wenli Xiao +4
We present Human to Humanoid (H2O), a reinforcement learning (RL) based framework that enables real-time whole-body teleoperation of a full-sized humanoid robot with only an RGB ca…
Multi-Object Tracking by Hierarchical Visual Representations
Jinkun Cao, Jiangmiao Pang, Kris Kitani
We propose a new visual hierarchical representation paradigm for multi-object tracking. It is more effective to discriminate between objects by attending to objects' compositional…
MGF: Mixed Gaussian Flow for Diverse Trajectory Prediction
Jiahe Chen, Jinkun Cao, Dahua Lin +2
To predict future trajectories, the normalizing flow with a standard Gaussian prior suffers from weak diversity. The ineffectiveness comes from the conflict between the fact of asy…
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Kristen Grauman, Andrew Westbury, Lorenzo Torresani +98
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric…
Universal Humanoid Motion Representations for Physics-Based Control
Zhengyi Luo, Jinkun Cao, Josh Merel +4
We present a universal motion representation that encompasses a comprehensive range of motor skills for physics-based humanoid control. Due to the high dimensionality of humanoids…