13 citations · 20 across the 13 of their papers we have counts for
13 papers
Retriever: Composing the Perception-Reasoning-Action Loop for Long-Horizon Manipulation
Linfeng Zhao, Haojie Huang, Jiayuan Mao +3
Building long-horizon robot agents requires composing closed-loop pipelines -- perception, belief update, planning, and control -- whose components run at different clocks and with…
APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts
Emily Jin, Joy Hsu, Yiqing Xu +3
Long-horizon robot planning requires jointly reasoning over semantic task structure and geometric feasibility. To successfully execute a task, a robot must decompose goals, select…
ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
Qineng Wang, Wenlong Huang, Yu Zhou +8
Embodied cognition argues that intelligence arises from sensorimotor interaction rather than passive observation. It raises an intriguing question: do modern vision-language models…
MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation
Chengshu Li, Mengdi Xu, Arpit Bahety +11
Imitation learning from large-scale, diverse human demonstrations has been shown to be effective for training robots, but collecting such data is costly and time-consuming. This ch…
Learning Compositional Behaviors from Demonstration and Language
Weiyu Liu, Neil Nie, Ruohan Zhang +2
We introduce Behavior from Language and Demonstration (BLADE), a framework for long-horizon robotic manipulation by integrating imitation learning and model-based planning. BLADE l…
LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models
Fan-Yun Sun, Weiyu Liu, Siyi Gu +6
Spatial reasoning is a fundamental aspect of human cognition, enabling intuitive understanding and manipulation of objects in three-dimensional space. While foundation models demon…
IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos
Yunong Liu, Cristobal Eyzaguirre, Manling Li +6
Shape assembly is a ubiquitous task in daily life, integral for constructing complex 3D structures like IKEA furniture. While significant progress has been made in developing auton…
MARPLE: A Benchmark for Long-Horizon Inference
Emily Jin, Zhuoyi Huang, Jan-Philipp Fränken +7
Reconstructing past events requires reasoning across long time horizons. To figure out what happened, we need to use our prior knowledge about the world and human behavior and draw…
Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
Manling Li, Shiyu Zhao, Qineng Wang +12
We aim to evaluate Large Language Models (LLMs) for embodied decision making. While a significant body of work has been leveraging LLMs for decision making in embodied environments…
Composable Part-Based Manipulation
Weiyu Liu, Jiayuan Mao, Joy Hsu +3
In this paper, we propose composable part-based manipulation (CPM), a novel approach that leverages object-part decomposition and part-part correspondences to improve learning and…
Learning Planning Abstractions from Language
Weiyu Liu, Geng Chen, Joy Hsu +2
This paper presents a framework for learning state and action abstractions in sequential decision-making domains. Our framework, planning abstraction from language (PARL), utilizes…
Naturally Supervised 3D Visual Grounding with Language-Regularized Concept Learners
Chun Feng, Joy Hsu, Weiyu Liu +1
3D visual grounding is a challenging task that often requires direct and dense supervision, notably the semantic label for each object in the scene. In this paper, we instead study…
Foundation Models in Robotics: Applications, Challenges, and the Future
Roya Firoozi, Johnathan Tucker, Stephen Tian +12
We survey applications of pretrained foundation models in robotics. Traditional deep learning models in robotics are trained on small datasets tailored for specific tasks, which li…