14 citations · 52 across the 14 of their papers we have counts for
16 papers
: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities
Physical Intelligence, Bo Ai, Ali Amin +85
We present a new robotic foundation model, called , that can enable strong out-of-the-box performance in a wide range of scenarios. can follow diverse language i…
MEM: Multi-Scale Embodied Memory for Vision Language Action Models
Marcel Torne, Karl Pertsch, Homer Walke +14
Conventionally, memory in end-to-end robotic learning involves inputting a sequence of past observations into the learned policy. However, in complex multi-stage real-world tasks,…
AsyncVLA: An Asynchronous VLA for Fast and Robust Navigation on the Edge
Noriaki Hirose, Catherine Glossop, Dhruv Shah +1
Robotic foundation models achieve strong generalization by leveraging internet-scale vision-language representations, but their massive computational cost creates a fundamental bot…
Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control
William Chen, Jagdeep Singh Bhatia, Catherine Glossop +6
Pretrained vision-language models (VLMs) can make semantic and visual inferences across diverse settings, providing valuable common-sense priors for robotic control. However, effec…
Training-Time Action Conditioning for Efficient Real-Time Chunking
Kevin Black, Allen Z. Ren, Michael Equi +1
Real-time chunking (RTC) enables vision-language-action models (VLAs) to generate smooth, reactive robot trajectories by asynchronously predicting action chunks and conditioning on…
: a VLA That Learns From Experience
Physical Intelligence, Ali Amin, Raichelle Aniceto +53
We study how vision-language-action (VLA) models can improve through real-world deployments via reinforcement learning (RL). We present a general-purpose method, RL with Experience…
OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation
Noriaki Hirose, Catherine Glossop, Dhruv Shah +1
Humans can flexibly interpret and compose different goal specifications, such as language instructions, spatial coordinates, or visual references, when navigating to a destination.…
CAST: Counterfactual Labels Improve Instruction Following in Vision-Language-Action Models
Catherine Glossop, William Chen, Arjun Bhorkar +2
Generalist robots should be able to understand and follow user instructions. Despite providing a powerful architecture for mapping open-vocabulary language instructions to robot ac…
LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos
Noriaki Hirose, Catherine Glossop, Ajay Sridhar +3
The world is filled with a wide variety of objects. For robots to be useful, they need the ability to find arbitrary objects described by people. In this paper, we present LeLaN(Le…
Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs
Hao-Tien Lewis Chiang, Zhuo Xu, Zipeng Fu +19
An elusive goal in navigation research is to build an intelligent agent that can understand multimodal instructions including natural language and image, and perform useful navigat…
SELFI: Autonomous Self-Improvement with Reinforcement Learning for Social Navigation
Noriaki Hirose, Dhruv Shah, Kyle Stachowicz +2
Autonomous self-improving robots that interact and improve with experience are key to the real-world deployment of robotic systems. In this paper, we propose an online learning met…
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
Soroush Nasiriany, Fei Xia, Wenhao Yu +20
Vision language models (VLMs) have shown impressive capabilities across a variety of tasks, from logical reasoning to visual understanding. This opens the door to richer interactio…
Chain of Code: Reasoning with a Language Model-Augmented Code Emulator
Chengshu Li, Jacky Liang, Andy Zeng +7
Code provides a general syntactic structure to build complex programs and perform precise computations when paired with a code interpreter - we hypothesize that language models (LM…
Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning
Dhruv Shah, Michael Equi, Blazej Osinski +3
Navigation in unfamiliar environments presents a major challenge for robots: while mapping and planning techniques can be used to build up a representation of the world, quickly di…
NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration
Ajay Sridhar, Dhruv Shah, Catherine Glossop +1
Robotic learning for navigation in unfamiliar environments needs to provide policies for both task-oriented navigation (i.e., reaching a goal that the robot has located), and task-…
Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents
Wenlong Huang, Fei Xia, Dhruv Shah +8
Recent progress in large language models (LLMs) has demonstrated the ability to learn and leverage Internet-scale knowledge through pre-training with autoregressive models. Unfortu…