Publications (38)
Practice Makes Perfect: Planning to Learn Skill Parameter Policies
Nishanth Kumar, Tom Silver, Willie McClinton +5
One promising approach towards effective robot decision making in complex, long-horizon tasks is to sequence together parameterized skills. We consider a setting where a robot is i…
Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics
Ken Kansky, Tom Silver, David A. Mély +7
The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress…
Unifying Deep Predicate Invention with Pre-trained Foundation Models
Qianwei Wang, Bowen Li, Zhanpeng Luo +6
Long-horizon robotic tasks are hard due to continuous state-action spaces and sparse feedback. Symbolic world models help by decomposing tasks into discrete predicates that capture…
GRACE: Generalizing Robot-Assisted Caregiving with User Functionality Embeddings
Ziang Liu, Yuanchen Ju, Yu Da +6
Robot caregiving should be personalized to meet the diverse needs of care recipients -- assisting with tasks as needed, while taking user agency in action into account. In physical…
Learning Symbolic Operators for Task and Motion Planning
Tom Silver, Rohan Chitnis, Joshua Tenenbaum +2
Robotic planning problems in hybrid state and action spaces can be solved by integrated task and motion planners (TAMP) that handle the complex interaction between motion-level dec…
Few-Shot Bayesian Imitation Learning with Logical Program Policies
Tom Silver, Kelsey R. Allen, Alex K. Lew +2
Humans can learn many novel tasks from a very small number (1--5) of demonstrations, in stark contrast to the data requirements of nearly tabula rasa deep learning methods. We prop…
A Human-in-the-Loop Confidence-Aware Failure Recovery Framework for Modular Robot Policies
Rohan Banerjee, Krishna Palempalli, Bohan Yang +5
Robots operating in unstructured human environments inevitably encounter failures, especially in robot caregiving scenarios. While humans can often help robots recover, excessive o…
FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalization
Rajat Kumar Jenamani, Tom Silver, Ben Dodson +7
Physical caregiving robots hold promise for improving the quality of life of millions worldwide who require assistance with feeding. However, in-home meal assistance remains challe…
GLIB: Efficient Exploration for Relational Model-Based Reinforcement Learning via Goal-Literal Babbling
Rohan Chitnis, Tom Silver, Joshua Tenenbaum +2
We address the problem of efficient exploration for transition model learning in the relational model-based reinforcement learning setting without extrinsic goals or rewards. Inspi…
Anticipatory Task and Motion Planning
Roshan Dhakal, Duc M. Nguyen, Tom Silver +2
We consider a sequential task and motion planning (tamp) setting in which a robot is assigned continuous-space rearrangement-style tasks one-at-a-time in an environment that persis…
Predicate Invention for Bilevel Planning
Tom Silver, Rohan Chitnis, Nishanth Kumar +4
Efficient planning in continuous state and action spaces is fundamentally hard, even when the transition model is deterministic and known. One way to alleviate this challenge is to…
Bilevel Learning for Bilevel Planning
Bowen Li, Tom Silver, Sebastian Scherer +1
A robot that learns from demonstrations should not just imitate what it sees -- it should understand the high-level concepts that are being demonstrated and generalize them to new…
CLAMP: Crowdsourcing a LArge-scale in-the-wild haptic dataset with an open-source device for Multimodal robot Perception
Pranav N. Thakkar, Shubhangi Sinha, Karan Baijal +11
Robust robot manipulation in unstructured environments often requires understanding object properties that extend beyond geometry, such as material or compliance-properties that ca…
CART-MPC: Coordinating Assistive Devices for Robot-Assisted Transferring with Multi-Agent Model Predictive Control
Ruolin Ye, Shuaixing Chen, Yunting Yan +6
Bed-to-wheelchair transferring is a ubiquitous activity of daily living (ADL), but especially challenging for caregiving robots with limited payloads. We develop a novel algorithm…
PDDLGym: Gym Environments from PDDL Problems
Tom Silver, Rohan Chitnis
We present PDDLGym, a framework that automatically constructs OpenAI Gym environments from PDDL domains and problems. Observations and actions in PDDLGym are relational, making the…
PrioriTouch: Adapting to User Contact Preferences for Whole-Arm Physical Human-Robot Interaction
Rishabh Madan, Jiawei Lin, Mahika Goel +10
Physical human-robot interaction (pHRI) requires robots to adapt to individual contact preferences, such as where and how much force is applied. Identifying preferences is difficul…
VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
Yichao Liang, Nishanth Kumar, Hao Tang +5
Broadly intelligent agents should form task-specific abstractions that selectively expose the essential elements of a task, while abstracting away the complexity of the raw sensori…
Learning Neuro-Symbolic Skills for Bilevel Planning
Tom Silver, Ashay Athalye, Joshua B. Tenenbaum +2
Decision-making is challenging in robotics environments with continuous object-centric states, continuous actions, long horizons, and sparse feedback. Hierarchical approaches, such…
Integrated Task and Motion Planning
Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay +4
The problem of planning for a robot that operates in environments containing a large number of objects, taking actions to move itself through the world as well as to change the sta…
SAVOR: Skill Affordance Learning from Visuo-Haptic Perception for Robot-Assisted Bite Acquisition
Zhanxin Wu, Bo Ai, Tom Silver +1
Robot-assisted feeding requires reliable bite acquisition, a challenging task due to the complex interactions between utensils and food with diverse physical properties. These inte…
Reinforcement Learning for Classical Planning: Viewing Heuristics as Dense Reward Generators
Clement Gehring, Masataro Asai, Rohan Chitnis +4
Recent advances in reinforcement learning (RL) have led to a growing interest in applying RL to classical planning domains or applying classical planning methods to some complex RL…
Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning
Rohan Chitnis, Tom Silver, Joshua B. Tenenbaum +2
In robotic domains, learning and planning are complicated by continuous state spaces, continuous action spaces, and long task horizons. In this work, we address these challenges wi…
Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures
Bowen Li, Mayank Mishra, Y. Isabel Liu +7
Intelligent robots should not only recover from failures, but also acquire the abstract knowledge needed to avoid them in the future. While reinforcement learning (RL) can learn re…
PG3: Policy-Guided Planning for Generalized Policy Generation
Ryan Yang, Tom Silver, Aidan Curtis +2
A longstanding objective in classical planning is to synthesize policies that generalize across multiple problems from the same domain. In this work, we study generalized policy se…
Residual Policy Learning
Tom Silver, Kelsey Allen, Josh Tenenbaum +1
We present Residual Policy Learning (RPL): a simple method for improving nondifferentiable policies using model-free deep reinforcement learning. RPL thrives in complex robotic man…
Coloring Between the Lines: Personalization in the Null Space of Planning Constraints
Tom Silver, Rajat Kumar Jenamani, Ziang Liu +2
Generalist robots must personalize in-the-wild to meet the diverse needs and preferences of long-term users. How can we enable flexible personalization without sacrificing safety o…
Online Bayesian Goal Inference for Boundedly-Rational Planning Agents
Tan Zhi-Xuan, Jordyn L. Mann, Tom Silver +2
People routinely infer the goals of others by observing their actions over time. Remarkably, we can do so even when those actions lead to failure, enabling us to assist others when…
ExoPredicator: Learning Abstract Models of Dynamic Worlds for Robot Planning
Yichao Liang, Dat Nguyen, Cambridge Yang +7
Long-horizon embodied planning is challenging because the world does not only change through an agent's actions: exogenous processes (e.g., water heating, dominoes cascading) unfol…
From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Models
Ashay Athalye, Nishanth Kumar, Tom Silver +4
Our aim is to learn to solve long-horizon decision-making problems in complex robotics domains given low-level skills and a handful of short-horizon demonstrations containing seque…
KinDER: A Physical Reasoning Benchmark for Robot Learning and Planning
Yixuan Huang, Bowen Li, Vaibhav Saxena +9
Robotic systems that interact with the physical world must reason about kinematic and dynamic constraints imposed by their own embodiment, their environment, and the task at hand.…
CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs
Rohan Chitnis, Tom Silver, Beomjoon Kim +2
Meta-planning, or learning to guide planning from experience, is a promising approach to improving the computational cost of planning. A general meta-planning strategy is to learn…
Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networks
Tom Silver, Rohan Chitnis, Aidan Curtis +3
Real-world planning problems often involve hundreds or even thousands of objects, straining the limits of modern planners. In this work, we address this challenge by learning to pr…
Seeing is Believing: Belief-Space Planning with Foundation Models as Uncertainty Estimators
Linfeng Zhao, Willie McClinton, Aidan Curtis +4
Generalizable robotic mobile manipulation in open-world environments poses significant challenges due to long horizons, complex goals, and partial observability. A promising approa…
Discovering State and Action Abstractions for Generalized Task and Motion Planning
Aidan Curtis, Tom Silver, Joshua B. Tenenbaum +2
Generalized planning accelerates classical planning by finding an algorithm-like policy that solves multiple instances of a task. A generalized plan can be learned from a few train…
SLAP: Shortcut Learning for Abstract Planning
Y. Isabel Liu, Bowen Li, Benjamin Eysenbach +1
Long-horizon decision-making with sparse rewards and continuous states and actions remains a fundamental challenge in AI and robotics. Task and motion planning (TAMP) is a model-ba…
Embodied Active Learning of Relational State Abstractions for Bilevel Planning
Amber Li, Tom Silver
State abstraction is an effective technique for planning in robotics environments with continuous states and actions, long task horizons, and sparse feedback. In object-oriented en…
Learning Efficient Abstract Planning Models that Choose What to Predict
Nishanth Kumar, Willie McClinton, Rohan Chitnis +3
An effective approach to solving long-horizon tasks in robotics domains with continuous state and action spaces is bilevel planning, wherein a high-level search over an abstraction…
Generalized Planning in PDDL Domains with Pretrained Large Language Models
Tom Silver, Soham Dan, Kavitha Srinivas +3
Recent work has considered whether large language models (LLMs) can function as planners: given a task, generate a plan. We investigate whether LLMs can serve as generalized planne…