74 citations · 131 across the 18 of their papers we have counts for
20 papers
VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes
Yen-Jen Wang, Jiaman Li, Sirui Chen +9
Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Learning this mapping requires synchronized egoc…
SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interaction
Sirui Chen, Shibo Zhao, Zhen Wu +3
Current humanoid reinforcement-learning policies excel at free-space motions but struggle with contact-rich tasks, as pure kinematic tracking cannot resolve the physical ambiguitie…
AnyLift: Scaling Motion Reconstruction from Internet Videos via 2D Diffusion
Hongjie Li, Heng Yu, Jiaman Li +4
Reconstructing 3D human motion and human-object interactions (HOI) from Internet videos is a fundamental step toward building large-scale datasets of human behavior. Existing metho…
WHOLE: World-Grounded Hand-Object Lifted from Egocentric Videos
Yufei Ye, Jiaman Li, Ryan Rong +1
Egocentric manipulation videos are highly challenging due to severe occlusions during interactions and frequent object entries and exits from the camera view as the person moves. C…
ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation
Hongjie Li, Hong-Xing Yu, Jiaman Li +1
Human-scene interaction (HSI) generation is crucial for applications in embodied AI, virtual reality, and robotics. Yet, existing methods cannot synthesize interactions in unseen e…
Lifting Motion to the 3D World via 2D Diffusion
Jiaman Li, C. Karen Liu, Jiajun Wu
Estimating 3D motion from 2D observations is a long-standing research challenge. Prior work typically requires training on datasets containing ground truth 3D motions, limiting the…
Human-Object Interaction from Human-Level Instructions
Zhen Wu, Jiaman Li, Pei Xu +1
Intelligent agents must autonomously interact with the environments to perform daily tasks based on human-level instructions. They need a foundational understanding of the world to…
Controllable Human-Object Interaction Synthesis
Jiaman Li, Alexander Clegg, Roozbeh Mottaghi +3
Synthesizing semantic-aware, long-horizon, human-object interaction is critical to simulate realistic human behaviors. In this work, we address the challenging problem of generatin…
Object Motion Guided Human Motion Synthesis
Jiaman Li, Jiajun Wu, C. Karen Liu
Modeling human behaviors in contextual environments has a wide range of applications in character animation, embodied AI, VR/AR, and robotics. In real-world scenarios, humans frequ…
Motion Question Answering via Modular Motion Programs
Mark Endo, Joy Hsu, Jiaman Li +1
In order to build artificial intelligence systems that can perceive and reason with human behavior in the real world, we must first design models that conduct complex spatio-tempor…
CIRCLE: Capture In Rich Contextual Environments
Joao Pedro Araujo, Jiaman Li, Karthik Vetrivel +5
Synthesizing 3D human motion in a contextual, ecological environment is important for simulating realistic activities people perform in the real world. However, conventional optics…
Scene Synthesis from Human Motion
Sifan Ye, Yixing Wang, Jiaman Li +4
Large-scale capture of human motion with diverse, complex scenes, while immensely useful, is often considered prohibitively costly. Meanwhile, human motion alone contains rich info…
Ego-Body Pose Estimation via Ego-Head Pose Estimation
Jiaman Li, C. Karen Liu, Jiajun Wu
Estimating 3D human motion from an egocentric video sequence plays a critical role in human behavior understanding and has various applications in VR/AR. However, naively learning…
GIMO: Gaze-Informed Human Motion Prediction in Context
Yang Zheng, Yanchao Yang, Kaichun Mo +5
Predicting human motion is critical for assistive robots and AR/VR applications, where the interaction with humans needs to be safe and comfortable. Meanwhile, an accurate predicti…
DenseGAP: Graph-Structured Dense Correspondence Learning with Anchor Points
Zhengfei Kuang, Jiaman Li, Mingming He +2
Establishing dense correspondence between two images is a fundamental computer vision problem, which is typically tackled by matching local feature descriptors. However, without gl…
Task-Generic Hierarchical Human Motion Prior using VAEs
Jiaman Li, Ruben Villegas, Duygu Ceylan +4
A deep generative model that describes human motions can benefit a wide range of fundamental computer vision and graphics tasks, such as providing robustness to video-based human p…
Dynamic Facial Asset and Rig Generation from a Single Scan
Jiaman Li, Zhengfei Kuang, Yajie Zhao +3
The creation of high-fidelity computer-generated (CG) characters used in film and gaming requires intensive manual labor and a comprehensive set of facial assets to be captured wit…
Learning to Generate Diverse Dance Motions with Transformer
Jiaman Li, Yihang Yin, Hang Chu +4
With the ongoing pandemic, virtual concerts and live events using digitized performances of musicians are getting traction on massive multiplayer online worlds. However, well chore…
VirtualHome: Simulating Household Activities via Programs
Xavier Puig, Kevin Ra, Marko Boben +4
In this paper, we are interested in modeling complex activities that occur in a typical household. We propose to use programs, i.e., sequences of atomic actions and interactions, a…
Learning to Act Properly: Predicting and Explaining Affordances from Images
Ching-Yao Chuang, Jiaman Li, Antonio Torralba +1
We address the problem of affordance reasoning in diverse scenes that appear in the real world. Affordances relate the agent's actions to their effects when taken on the surroundin…