8 papers
Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim
Ivan Kapelyukh, Yafei Hu, Ran Gong +6
Spatial reasoning is fundamental to general robot intelligence, as it enables robots to complete long-horizon tasks involving multi-object interaction. We introduce Simify, a train…
Efficient Sim-to-Real Transfer of World-Action Models from Synthetic Priors
Zixing Wang, Kausik Sivakumar, Jinghuan Shang +5
Bridging the sim-to-real gap is a core challenge in deploying learned manipulation policies. Sim-to-real learning is attractive because it can replace expensive real robot demonstr…
Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks
Rutav Shah, Rajat Kumar Jenamani, Xiaohan Zhang +5
Many robotic tasks require short-term memory, whether it's retrieving an object that's no longer visible or turning off an appliance after a set period. Yet, most visuomotor polici…
ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priors
Zifan Xu, Ran Gong, Maria Vittoria Minniti +10
Learning generalizable and robust behavior cloning policies requires large volumes of high-quality robotics data. While human demonstrations (e.g., through teleoperation) serve as…
You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vector
Omkar Patil, Ondrej Biza, Thomas Weng +9
Generative robot policies trained on demonstrations using behavior cloning often learn actions that are sub-optimal or misaligned with respect to the downstream task. Policy improv…
AnyTask: an Automated Task and Data Generation Framework for Advancing Sim-to-Real Policy Learning
Ran Gong, Xiaohan Zhang, Jinghuan Shang +11
Generalist robot learning remains constrained by data: large-scale, diverse, and high-quality interaction data are expensive to collect in the real world. While simulation has beco…
Data-Efficient Multitask DAgger
Haotian Fu, Ran Gong, Xiaohan Zhang +3
Generalist robot policies that can perform many tasks typically require extensive expert data or simulations for training. In this work, we propose a novel Data-Efficient multitask…
ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation
Suning Huang, Qianzhong Chen, Xiaohan Zhang +2
3D world models (i.e., learning-based 3D dynamics models) offer a promising approach to generalizable robotic manipulation by capturing the underlying physics of environment evolut…