52 citations · 91 across the 11 of their papers we have counts for
9 papers · 1 filter
Bridging Perception and Action: Spatially-Grounded Mid-Level Representations for Robot Generalization
Jonathan Yang, Chuyuan Kelly Fu, Dhruv Shah +3
In this work, we investigate how spatially grounded auxiliary representations can provide both broad, high-level grounding as well as direct, actionable information to improve poli…
Vision Language Models are In-Context Value Learners
Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid +15
Predicting temporal progress from visual trajectories is important for intelligent robots that can learn, adapt, and improve. However, learning such progress estimator, or temporal…
Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta +7
How can robot manipulation policies generalize to novel tasks involving unseen object types and new motions? In this paper, we provide a solution in terms of predicting motion info…
VADER: Visual Affordance Detection and Error Recovery for Multi Robot Human Collaboration
Michael Ahn, Montserrat Gonzalez Arenas, Matthew Bennice +22
Robots today can exploit the rich world knowledge of large language models to chain simple behavioral skills into long-horizon tasks. However, robots often get interrupted during l…
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Soroush Nasiriany, Fei Xia, Wenhao Yu +20
Vision language models (VLMs) have shown impressive capabilities across a variety of tasks, from logical reasoning to visual understanding. This opens the door to richer interactio…
Learning to Learn Faster from Human Feedback with Language Model Predictive Control
Jacky Liang, Fei Xia, Wenhao Yu +47
Large language models (LLMs) have been shown to exhibit a wide range of capabilities, such as writing robot code from language commands -- enabling non-experts to direct robot beha…
Generative Expressive Robot Behaviors using Large Language Models
Karthik Mahadevan, Jonathan Chien, Noah Brown +6
People employ expressive behaviors to effectively communicate and coordinate their actions with others, such as nodding to acknowledge a person glancing at them or saying "excuse m…
Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections
Lihan Zha, Yuchen Cui, Li-Heng Lin +5
Today's robot policies exhibit subpar performance when faced with the challenge of generalizing to novel environments. Human corrective feedback is a crucial form of guidance to en…
Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents
Wenlong Huang, Fei Xia, Dhruv Shah +8
Recent progress in large language models (LLMs) has demonstrated the ability to learn and leverage Internet-scale knowledge through pre-training with autoregressive models. Unfortu…