141 citations · 399 across the 34 of their papers we have counts for
37 papers
DemoStream: Batched Task-and-Motion-Plan Optimization for Automated Learning from Demonstration
Caelan Garrett, Neel Jawale, Fabio Ramos
Learning from demonstration is an effective strategy for endowing robots with manipulation capabilities but requires training on datasets comprised of massive amounts of demonstrat…
Human2Any: Human-to-Robot Transfer via Constraint-Aware Compositional Planning
Shuo Cheng, Chuye Zhang, Alfred Cueva +3
Human videos are a scalable source of supervision for robot manipulation, as they are abundant and naturally capture rich object interactions. However, transferring human demonstra…
HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning
Kevin Lin, Ajay Mandlekar, Caelan Reed Garrett +7
Imitation learning is a promising approach for training humanoid robots to both walk and manipulate, but it requires a large number of demonstrations, which are time-intensive and…
ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning
Zihan Zhou, Animesh Garg, Ajay Mandlekar +1
Long-horizon manipulation has been a long-standing challenge in the robotics community. We propose ReinforceGen, a system that combines task decomposition, data generation, imitati…
ScheduleStream: Temporal Planning with Samplers for GPU-Accelerated Multi-Arm Task and Motion Planning & Scheduling
Caelan Garrett, Fabio Ramos
Bimanual and humanoid robots are appealing because of their human-like ability to leverage multiple arms to efficiently complete tasks. However, controlling multiple arms at once i…
The Reality Gap in Robotics: Challenges, Solutions, and Best Practices
Elie Aljalbout, Jiaxu Xing, Angel Romero +9
Machine learning has facilitated significant advancements across various robotics domains, including navigation, locomotion, and manipulation. Many such achievements have been driv…
Learning to Plan & Schedule with Reinforcement-Learned Bimanual Robot Skills
Weikang Wan, Fabio Ramos, Xuning Yang +1
Long-horizon contact-rich bimanual manipulation presents a significant challenge, requiring complex coordination involving a mixture of parallel execution and sequential collaborat…
Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training
Shuo Cheng, Liqian Ma, Zhenyang Chen +3
Behavior cloning has shown promise for robot manipulation, but real-world demonstrations are costly to acquire at scale. While simulated data offers a scalable alternative, particu…
HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation
Yi Li, Yuquan Deng, Jesse Zhang +9
Large foundation models have shown strong open-world generalization to complex problems in vision and language, but similar levels of generalization have yet to be achieved in robo…
Differentiable GPU-Parallelized Task and Motion Planning
William Shen, Caelan Garrett, Nishanth Kumar +5
Planning long-horizon robot manipulation requires making discrete decisions about which objects to interact with and continuous decisions about how to interact with them. A robot p…
Open-World Task and Motion Planning via Vision-Language Model Generated Constraints
Nishanth Kumar, William Shen, Fabio Ramos +4
Foundation models like Vision-Language Models (VLMs) excel at common sense vision and language tasks such as visual question answering. However, they cannot yet directly solve comp…
SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
Caelan Garrett, Ajay Mandlekar, Bowen Wen +1
Imitation learning from human demonstrations is an effective paradigm for robot manipulation, but acquiring large datasets is costly and resource-intensive, especially for long-hor…
SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
Zihan Zhou, Animesh Garg, Dieter Fox +2
Robot learning has proven to be a general and effective technique for programming manipulators. Imitation learning is able to teach robots solely from human demonstrations but is b…
Guiding Long-Horizon Task and Motion Planning with Vision Language Models
Zhutian Yang, Caelan Garrett, Dieter Fox +2
Vision-Language Models (VLM) can generate plausible high-level plans when prompted with a goal, the context, an image of the scene, and any planning constraints. However, there is…
IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning
Ryan Hoque, Ajay Mandlekar, Caelan Garrett +2
Imitation learning is a promising paradigm for training robot control policies, but these policies can suffer from distribution shift, where the conditions at evaluation time diffe…
cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation
Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman +9
This paper explores the problem of collision-free motion generation for manipulators by formulating it as a global motion optimization problem. We develop a parallel optimization t…
NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors
Shuo Cheng, Caelan Garrett, Ajay Mandlekar +1
Solving complex manipulation tasks in household and factory settings remains challenging due to long-horizon reasoning, fine-grained interactions, and broad object and scene divers…
Human-in-the-Loop Task and Motion Planning for Imitation Learning
Ajay Mandlekar, Caelan Garrett, Danfei Xu +1
Imitation learning from human demonstrations can teach robots complex manipulation skills, but is time-consuming and labor intensive. In contrast, Task and Motion Planning (TAMP) s…
Meta-Policy Learning over Plan Ensembles for Robust Articulated Object Manipulation
Constantinos Chamzas, Caelan Garrett, Balakumar Sundaralingam +2
Recent work has shown that complex manipulation skills, such as pushing or pouring, can be learned through state-of-the-art learning based techniques, such as Reinforcement Learnin…
DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability
Xiaolin Fang, Caelan Reed Garrett, Clemens Eppner +3
Generative models such as diffusion models, excel at capturing high-dimensional distributions with diverse input modalities, e.g. robot trajectories, but are less effective at mult…
Imitating Task and Motion Planning with Visuomotor Transformers
Murtaza Dalal, Ajay Mandlekar, Caelan Garrett +3
Imitation learning is a powerful tool for training robot manipulation policies, allowing them to learn from expert demonstrations without manual programming or trial-and-error. How…
Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning
Zhutian Yang, Caelan Reed Garrett, Tomás Lozano-Pérez +2
We present a learning-enabled Task and Motion Planning (TAMP) algorithm for solving mobile manipulation problems in environments with many articulated and movable obstacles. Our id…
Cooperative Task and Motion Planning for Multi-Arm Assembly Systems
Jingkai Chen, Jiaoyang Li, Yijiang Huang +7
Multi-robot assembly systems are becoming increasingly appealing in manufacturing due to their ability to automatically, flexibly, and quickly construct desired structural designs.…
Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordances
Aidan Curtis, Xiaolin Fang, Leslie Pack Kaelbling +2
We present a strategy for designing and building very general robot manipulation systems involving the integration of a general-purpose task-and-motion planner with engineered and…