Publications (28)
Learning to Bridge the Gap: Efficient Novelty Recovery with Planning and Reinforcement Learning
Alicia Li, Nishanth Kumar, Tomás Lozano-Pérez +1
The real world is unpredictable. Therefore, to solve long-horizon decision-making problems with autonomous robots, we must construct agents that are capable of adapting to changes…
Practice Makes Perfect: Planning to Learn Skill Parameter Policies
Nishanth Kumar, Tom Silver, Willie McClinton +5
One promising approach towards effective robot decision making in complex, long-horizon tasks is to sequence together parameterized skills. We consider a setting where a robot is i…
Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures
Bowen Li, Mayank Mishra, Y. Isabel Liu +7
Intelligent robots should not only recover from failures, but also acquire the abstract knowledge needed to avoid them in the future. While reinforcement learning (RL) can learn re…
Guided Exploration for Efficient Relational Model Learning
Annie Feng, Nishanth Kumar, Tomas Lozano-Perez +1
Efficient exploration is critical for learning relational models in large-scale environments with complex, long-horizon tasks. Random exploration methods often collect redundant or…
Preference-Conditioned Language-Guided Abstraction
Andi Peng, Andreea Bobu, Belinda Z. Li +5
Learning from demonstrations is a common way for users to teach robots, but it is prone to spurious feature correlations. Recent work constructs state abstractions, i.e. visual rep…
Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control
Jonathan Chang, Nishanth Kumar, Sean Hastings +6
Robots need to learn skills that can not only generalize across similar problems but also be directed to a specific goal. Previous methods either train a new skill for every differ…
The Past and Present of Imitation Learning: A Citation Chain Study
Nishanth Kumar
Imitation Learning is a promising area of active research. Over the last 30 years, Imitation Learning has advanced significantly and been used to solve difficult tasks ranging from…
Task Scoping: Generating Task-Specific Abstractions for Planning in Open-Scope Models
Michael Fishman, Nishanth Kumar, Cameron Allen +4
A general-purpose planning agent requires an open-scope world model: one rich enough to tackle any of the wide range of tasks it may be asked to solve over its operational lifetime…
Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfaction
Aidan Curtis, Nishanth Kumar, Jing Cao +2
Recent developments in pretrained large language models (LLMs) applied to robotics have demonstrated their capacity for sequencing a set of discrete skills to achieve open-ended go…
From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Models
Ashay Athalye, Nishanth Kumar, Tom Silver +4
Our aim is to learn to solve long-horizon decision-making problems in complex robotics domains given low-level skills and a handful of short-horizon demonstrations containing seque…
KinDER: A Physical Reasoning Benchmark for Robot Learning and Planning
Yixuan Huang, Bowen Li, Vaibhav Saxena +9
Robotic systems that interact with the physical world must reason about kinematic and dynamic constraints imposed by their own embodiment, their environment, and the task at hand.…
Open-World Task and Motion Planning via Vision-Language Model Generated Constraints
Nishanth Kumar, William Shen, Fabio Ramos +4
Foundation models like Vision-Language Models (VLMs) excel at common sense vision and language tasks such as visual question answering. However, they cannot yet directly solve comp…
Predicate Invention for Bilevel Planning
Tom Silver, Rohan Chitnis, Nishanth Kumar +4
Efficient planning in continuous state and action spaces is fundamentally hard, even when the transition model is deterministic and known. One way to alleviate this challenge is to…
MinePlanner: A Benchmark for Long-Horizon Planning in Large Minecraft Worlds
William Hill, Ireton Liu, Anita De Mello Koch +4
We propose a new benchmark for planning tasks based on the Minecraft game. Our benchmark contains 45 tasks overall, but also provides support for creating both propositional and nu…
PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAX
Guangyao Zhou, Antoine Dedieu, Nishanth Kumar +4
PGMax is an open-source Python package for (a) easily specifying discrete Probabilistic Graphical Models (PGMs) as factor graphs; and (b) automatically running efficient and scalab…
AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation
Jiafei Duan, Wilbert Pumacay, Nishanth Kumar +7
Robotic manipulation in open-world settings requires not only task execution but also the ability to detect and learn from failures. While recent advances in vision-language models…
Income distribution and inequality in India: 2014-19
Anand Sahasranaman, Nishanth Kumar
We study the evolution of income in India from 2014-19 and find that while income inequality remains largely consistent over this time, the lower end of the income distribution has…
Differentiable GPU-Parallelized Task and Motion Planning
William Shen, Caelan Garrett, Nishanth Kumar +5
Planning long-horizon robot manipulation requires making discrete decisions about which objects to interact with and continuous decisions about how to interact with them. A robot p…
Network structure of COVID-19 spread and the lacuna in India's testing strategy
Anand Sahasranaman, Nishanth Kumar
We characterize the network of COVID-19 spread in India and find that the transmission rate is 0.43, with daily case growth driven by individuals who contracted the virus abroad. W…
Seeing is Believing: Belief-Space Planning with Foundation Models as Uncertainty Estimators
Linfeng Zhao, Willie McClinton, Aidan Curtis +4
Generalizable robotic mobile manipulation in open-world environments poses significant challenges due to long horizons, complex goals, and partial observability. A promising approa…
Follow the Signs: Using Textual Cues and LLMs to Guide Efficient Robot Navigation
Jing Cao, Nishanth Kumar, Aidan Curtis
Autonomous navigation in unfamiliar environments often relies on geometric mapping and planning strategies that overlook rich semantic cues such as signs, room numbers, and textual…
Urbanization, economic development, and income distribution dynamics in India
Anand Sahasranaman, Nishanth Kumar, Luis M. A. Bettencourt
India's urbanization is often characterized as particularly challenging and very unequal but systematic empirical analyses, comparable to other nations, have largely been lacking.…
VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
Yichao Liang, Nishanth Kumar, Hao Tang +5
Broadly intelligent agents should form task-specific abstractions that selectively expose the essential elements of a task, while abstracting away the complexity of the raw sensori…
TiPToP: A Modular Open-Vocabulary Robot Manipulation System That Plans
William Shen, Nishanth Kumar, Sahit Chintalapudi +8
We present TiPToP, a modular manipulation system that integrates pretrained foundation models with a GPU-accelerated Task and Motion Planner to solve tasks directly from RGB images…
Learning Efficient Abstract Planning Models that Choose What to Predict
Nishanth Kumar, Willie McClinton, Rohan Chitnis +3
An effective approach to solving long-horizon tasks in robotics domains with continuous state and action spaces is bilevel planning, wherein a high-level search over an abstraction…
Adaptive Language-Guided Abstraction from Contrastive Explanations
Andi Peng, Belinda Z. Li, Ilia Sucholutsky +4
Many approaches to robot learning begin by inferring a reward function from a set of human demonstrations. To learn a good reward, it is necessary to determine which features of th…
Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes
Sean Segal, Nishanth Kumar, Sergio Casas +4
Self-driving vehicles must perceive and predict the future positions of nearby actors in order to avoid collisions and drive safely. A learned deep learning module is often respons…
Rate of Change Analysis for Interestingness Measures
Nandan Sudarsanam, Nishanth Kumar, Abhishek Sharma +1
The use of Association Rule Mining techniques in diverse contexts and domains has resulted in the creation of numerous interestingness measures. This, in turn, has motivated resear…