356 citations · 1.5k across the 30 of their papers we have counts for
10 papers · 1 filter
Interactive Language: Talking to Robots in Real Time
Corey Lynch, Ayzaan Wahid, Jonathan Tompson +5
We present a framework for building interactive, real-time, natural language-instructable robots in the real world, and we open source related assets (dataset, environment, benchma…
VIRDO++: Real-World, Visuo-tactile Dynamics and Perception of Deformable Objects
Youngsun Wi, Andy Zeng, Pete Florence +1
Deformable objects manipulation can benefit from representations that seamlessly integrate vision and touch while handling occlusions. In this work, we present a novel approach for…
Code as Policies: Language Model Programs for Embodied Control
Jacky Liang, Wenlong Huang, Fei Xia +5
Large language models (LLMs) trained on code completion have been shown to be capable of synthesizing simple Python programs from docstrings [1]. We find that these code-writing LL…
Inner Monologue: Embodied Reasoning through Planning with Language Models
Wenlong Huang, Fei Xia, Ted Xiao +14
Recent works have shown how the reasoning capabilities of Large Language Models (LLMs) can be applied to domains beyond natural language processing, such as planning and interactio…
Reinforcement Learning with Neural Radiance Fields
Danny Driess, Ingmar Schubert, Pete Florence +2
It is a long-standing problem to find effective representations for training reinforcement learning (RL) agents. This paper demonstrates that learning state representations with su…
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
Andy Zeng, Maria Attarian, Brian Ichter +10
Large pretrained (e.g., "foundation") models exhibit distinct capabilities depending on the domain of data they are trained on. While these domains are generic, they may only barel…