2 citations · 4 across the 19 of their papers we have counts for
20 papers
RAPID: Robot Agentic Programming from Demonstrations
Yuyao Liu, Jiayuan Mao, David Hsu +2
Coding agents have demonstrated enormous success in solving complex programming problems. To leverage their potential for robot systems, this work introduces Robot Agentic Programm…
TiPToP: A Modular Open-Vocabulary Robot Manipulation System That Plans
William Shen, Nishanth Kumar, Sahit Chintalapudi +8
We present TiPToP, a modular manipulation system that integrates pretrained foundation models with a GPU-accelerated Task and Motion Planner to solve tasks directly from RGB images…
Rational Inverse Reasoning: Few-Shot Imitation by Inferring Intent through Planning
Ben Zandonati, Tomás Lozano-Pérez, Leslie Pack Kaelbling
Humans can learn a new manipulation task from one or two demonstrations and then perform it in a new room, with new objects, under new constraints. Modern robot imitation learning,…
LLM-Guided Probabilistic Program Induction for POMDP Model Estimation
Aidan Curtis, Hao Tang, Thiago Veloso +4
Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty. While there are many approaches to approximately solving POMDPs, we aim to address…
Streaming Flow Policy: Simplifying diffusion/flow-matching policies by treating action trajectories as flow trajectories
Sunshine Jiang, Xiaolin Fang, Nicholas Roy +3
Recent advances in diffusionflow-matching policies have enabled imitation learning of complex, multi-modal action trajectories. However, they are computationally expensive becau…
From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Models
Ashay Athalye, Nishanth Kumar, Tom Silver +4
Our aim is to learn to solve long-horizon decision-making problems in complex robotics domains given low-level skills and a handful of short-horizon demonstrations containing seque…
One-Shot Manipulation Strategy Learning by Making Contact Analogies
Yuyao Liu, Jiayuan Mao, Joshua Tenenbaum +2
We present a novel approach, MAGIC (manipulation analogies for generalizable intelligent contacts), for one-shot learning of manipulation strategies with fast and extensive general…
Keypoint Abstraction using Large Models for Object-Relative Imitation Learning
Xiaolin Fang, Bo-Ruei Huang, Jiayuan Mao +4
Generalization to novel object configurations and instances across diverse tasks and environments is a critical challenge in robotics. Keypoint-based representations have been prov…
Combining Planning and Diffusion for Mobility with Unknown Dynamics
Yajvan Ravan, Zhutian Yang, Tao Chen +2
Manipulation of large objects over long horizons (such as carts in a warehouse) is an essential skill for deployable robotic systems. Large objects require mobile manipulation whic…
Guiding Long-Horizon Task and Motion Planning with Vision Language Models
Zhutian Yang, Caelan Garrett, Dieter Fox +2
Vision-Language Models (VLM) can generate plausible high-level plans when prompted with a goal, the context, an image of the scene, and any planning constraints. However, there is…
SceneComplete: Open-World 3D Scene Completion in Cluttered Real World Environments for Robot Manipulation
Aditya Agarwal, Gaurav Singh, Bipasha Sen +2
Careful robot manipulation in every-day cluttered environments requires an accurate understanding of the 3D scene, in order to grasp and place objects stably and reliably and to av…
Learning to Bridge the Gap: Efficient Novelty Recovery with Planning and Reinforcement Learning
Alicia Li, Nishanth Kumar, Tomás Lozano-Pérez +1
The real world is unpredictable. Therefore, to solve long-horizon decision-making problems with autonomous robots, we must construct agents that are capable of adapting to changes…
Embodied Uncertainty-Aware Object Segmentation
Xiaolin Fang, Leslie Pack Kaelbling, Tomás Lozano-Pérez
We introduce uncertainty-aware object instance segmentation (UncOS) and demonstrate its usefulness for embodied interactive segmentation. To deal with uncertainty in robot percepti…
Towards Practical Finite Sample Bounds for Motion Planning in TAMP
Seiji Shaw, Aidan Curtis, Leslie Pack Kaelbling +2
When using sampling-based motion planners, such as PRMs, in configuration spaces, it is difficult to determine how many samples are required for the PRM to find a solution consiste…
Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction
Aidan Curtis, Nishanth Kumar, Jing Cao +2
Recent developments in pretrained large language models (LLMs) applied to robotics have demonstrated their capacity for sequencing a set of discrete skills to achieve open-ended go…
Partially Observable Task and Motion Planning with Uncertainty and Risk Awareness
Aidan Curtis, George Matheos, Nishad Gothoskar +4
Integrated task and motion planning (TAMP) has proven to be a valuable approach to generalizable long-horizon robotic manipulation and navigation problems. However, the typical TAM…
Practice Makes Perfect: Planning to Learn Skill Parameter Policies
Nishanth Kumar, Tom Silver, Willie McClinton +5
One promising approach towards effective robot decision making in complex, long-horizon tasks is to sequence together parameterized skills. We consider a setting where a robot is i…
What Planning Problems Can A Relational Neural Network Solve?
Jiayuan Mao, Tomás Lozano-Pérez, Joshua B. Tenenbaum +1
Goal-conditioned policies are generally understood to be "feed-forward" circuits, in the form of neural networks that map from the current state and the goal specification to the n…
Learning Reusable Manipulation Strategies
Jiayuan Mao, Joshua B. Tenenbaum, Tomás Lozano-Pérez +1
Humans demonstrate an impressive ability to acquire and generalize manipulation "tricks." Even from a single demonstration, such as using soup ladles to reach for distant objects,…
Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning
Zhutian Yang, Caelan Reed Garrett, Tomás Lozano-Pérez +2
We present a learning-enabled Task and Motion Planning (TAMP) algorithm for solving mobile manipulation problems in environments with many articulated and movable obstacles. Our id…