565 citations · 571 across the 10 of their papers we have counts for
13 papers
Code to Control: Synthesizing Parameterized Reactive Controllers
Zergham Ahmed, Joshua B. Tenenbaum, Chris Bates +1
Recent LLM-based approaches to control either invoke a language model to select actions or synthesize world models that require planning at every decision, introducing latency that…
EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning
Yichao Liang, Amber Li, Dat Nguyen +13
A robot should be able to learn through experiments how unfamiliar objects behave and interact, then plan with that knowledge. It need not start from scratch: physics engines suppl…
Induction and Inquiry via Probabilistic Reasoning over Language and Code
Wasu Top Piriyakulkij, Sam Acquaviva, Cassidy Langenfeld +2
How humans grow and maintain abstract knowledge from the sparse, streaming noisy data of experience is a longstanding challenge in cognitive science. Any computational account must…
DiG-bench: Discovery in Games
Ruairidh M. Battleday, Kai Sandbrink, Jimi Cullen-Drohan +13
Discovery---formulating novel generalizations---is a central part of the scientific process. Despite its importance, there is a gap in the current AI benchmark landscape, with few…
Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents
Zergham Ahmed, Kazuki Irie, Joshua B. Tenenbaum +2
Humans learn abstractions and use them to plan efficiently to quickly generalize across tasks -- an ability that remains challenging for state-of-the-art large language model (LLM)…
ExoPredicator: Learning Abstract Models of Dynamic Worlds for Robot Planning
Yichao Liang, Dat Nguyen, Cambridge Yang +7
Long-horizon embodied planning is challenging because the world does not only change through an agent's actions: exogenous processes (e.g., water heating, dominoes cascading) unfol…
LLM-Guided Probabilistic Program Induction for POMDP Model Estimation
Aidan Curtis, Hao Tang, Thiago Veloso +4
Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty. While there are many approaches to approximately solving POMDPs, we aim to address…
Synthesizing world models for bilevel planning
Zergham Ahmed, Joshua B. Tenenbaum, Christopher J. Bates +1
Modern reinforcement learning (RL) systems have demonstrated remarkable capabilities in complex environments, such as video games. However, they still fall short of achieving human…
Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Models
Luca M. Schulze Buschoff, Konstantinos Voudouris, Elif Akata +3
Pre-trained vision language models still fall short of human visual cognition. In an effort to improve visual cognition and align models with human behavior, we introduce visual st…
Keypoint Abstraction using Large Models for Object-Relative Imitation Learning
Xiaolin Fang, Bo-Ruei Huang, Jiayuan Mao +4
Generalization to novel object configurations and instances across diverse tasks and environments is a critical challenge in robotics. Keypoint-based representations have been prov…
VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
Yichao Liang, Nishanth Kumar, Hao Tang +5
Broadly intelligent agents should form task-specific abstractions that selectively expose the essential elements of a task, while abstracting away the complexity of the raw sensori…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Map Induction: Compositional spatial submap learning for efficient exploration in novel environments
Sugandha Sharma, Aidan Curtis, Marta Kryven +2
Humans are expert explorers. Understanding the computational cognitive mechanisms that support this efficiency can advance the study of the human mind and enable more efficient exp…