5 papers
Retriever: Composing Closed-Loop Asynchronous Robot Programs
Linfeng Zhao, Haojie Huang, Jiayuan Mao +3
Building long-horizon robot agents requires composing closed-loop pipelines -- perception, belief update, planning, and control -- whose components run at different clocks and with…
Learning Compositional Behaviors from Demonstration and Language
Weiyu Liu, Neil Nie, Ruohan Zhang +2
We introduce Behavior from Language and Demonstration (BLADE), a framework for long-horizon robotic manipulation by integrating imitation learning and model-based planning. BLADE l…
CLEVRER-Humans: Describing Physical and Causal Events the Human Way
Jiayuan Mao, Xuelin Yang, Xikun Zhang +2
Building machines that can reason about physical events and their causal relationships is crucial for flexible interaction with the physical world. However, most existing physical…
What Makes a Maze Look Like a Maze?
Joy Hsu, Jiayuan Mao, Joshua B. Tenenbaum +2
A unique aspect of human visual understanding is the ability to flexibly interpret abstract concepts: acquiring lifted rules explaining what they symbolize, grounding them across f…
Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
Manling Li, Shiyu Zhao, Qineng Wang +12
We aim to evaluate Large Language Models (LLMs) for embodied decision making. While a significant body of work has been leveraging LLMs for decision making in embodied environments…