1 citations · 1 across the 13 of their papers we have counts for
12 papers · 1 filter
World Model for Robot Learning: A Comprehensive Survey
Bohan Hou, Gen Li, Jindou Jia +15
World models, which are predictive representations of how environments evolve under actions, have become a central component of robot learning. They support policy learning, planni…
D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping
Haozhe Lou, Mingtong Zhang, Haoran Geng +9
Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real…
DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
Shenyuan Gao, William Liang, Kaiyuan Zheng +27
Being able to simulate the outcomes of actions in varied environments will revolutionize the development of generalist agents at scale. However, modeling these world dynamics, espe…
Large Video Planner Enables Generalizable Robot Control
Boyuan Chen, Tianyuan Zhang, Haoran Geng +9
General-purpose robots require decision-making models that generalize across diverse tasks and environments. Recent works build robot foundation models by extending multimodal larg…
DIPOLE: Fusing Vision and Geometry for Robust Visuomotor Generalization
Yikai Tang, Haoran Geng, Jindou Jia +5
Imitation learning has emerged as a crucial approach for acquiring visuomotor skills from demonstrations, where designing effective observation encoders is essential for policy gen…
End-to-end RL Improves Dexterous Grasping Policies
Ritvik Singh, Karl Van Wyk, Pieter Abbeel +3
This work explores techniques to scale up image-based end-to-end learning for dexterous grasping with an arm + hand system. Unlike state-based RL, vision-based RL is much more memo…
Deep Sensorimotor Control by Imitating Predictive Models of Human Motion
Himanshu Gaurav Singh, Pieter Abbeel, Jitendra Malik +1
As the embodiment gap between a robot and a human narrows, new opportunities arise to leverage datasets of humans interacting with their surroundings for robot learning. We propose…
The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio
Renhao Wang, Haoran Geng, Tingle Li +7
Robots must integrate multiple sensory modalities to act effectively in the real world. Yet, learning such multimodal policies at scale remains challenging. Simulation offers a via…
SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blending
Yuxuan Kuang, Haoran Geng, Amine Elhafsi +5
Humanoid robots hold significant potential in accomplishing daily tasks across diverse environments thanks to their flexibility and human-like morphology. Recent works have made si…
ViTacFormer: Learning Cross-Modal Representation for Visuo-Tactile Dexterous Manipulation
Liang Heng, Haoran Geng, Kaifeng Zhang +2
Dexterous manipulation is a cornerstone capability for robotic systems aiming to interact with the physical world in a human-like manner. Although vision-based methods have advance…
Rodrigues Network for Learning Robot Actions
Jialiang Zhang, Haoran Geng, Yang You +4
Understanding and predicting articulated actions is important in robot learning. However, common architectures such as MLPs and Transformers lack inductive biases that reflect the…
DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy
Yuran Wang, Ruihai Wu, Yue Chen +7
Garment manipulation is a critical challenge due to the diversity in garment categories, geometries, and deformations. Despite this, humans can effortlessly handle garments, thanks…