103 citations · 127 across the 12 of their papers we have counts for
14 papers · 1 filter
Memory Without Curation: Learning Multi-Stream Latent Memory for Robot Policies
Yilin Wu, Rutav Shah, Shuaiyi Huang +5
It is widely appreciated that robotic policies require long-term memory to perform complex tasks. However, directly conditioning policies on all past observations leaves policy lea…
Memory Retrieval in Visuomotor Policies for Long-Horizon Robot Control
Rutav Shah, Yisu Li, Femi Bello +2
General-purpose robots operating in partially observable environments, such as homes, require memory to support autonomy. They must recall diverse information from the past, such a…
Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks
Rutav Shah, Rajat Kumar Jenamani, Xiaohan Zhang +5
Many robotic tasks require short-term memory, whether it's retrieving an object that's no longer visible or turning off an appliance after a set period. Yet, most visuomotor polici…
MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos
Rutav Shah, Shuijing Liu, Qi Wang +5
We aim to enable humanoid robots to efficiently solve new manipulation tasks from a few video examples. In-context learning (ICL) is a promising framework for achieving this goal d…
Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models
Huihan Liu, Rutav Shah, Shuijing Liu +6
Assistive teleoperation, where control is shared between a human and a robot, enables efficient and intuitive human-robot collaboration in diverse and unstructured environments. A…
SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning
Yu Zhang, Yuqi Xie, Huihan Liu +4
Imitation learning advances robot capabilities by enabling the acquisition of diverse behaviors from human demonstrations. However, large-scale datasets used for policy training of…
Towards Embodiment Scaling Laws in Robot Locomotion
Bo Ai, Liu Dai, Nico Bohlinger +7
Cross-embodiment generalization underpins the vision of building generalist embodied agents for any robot, yet its enabling factors remain poorly understood. We investigate embodim…
RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning
Charles Xu, Qiyang Li, Jianlan Luo +1
Recent advances in robotic foundation models have enabled the development of generalist policies that can adapt to diverse tasks. While these models show impressive flexibility, th…
BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulation
Rutav Shah, Albert Yu, Yifeng Zhu +2
To operate at a building scale, service robots must perform very long-horizon mobile manipulation tasks by navigating to different rooms, accessing different floors, and interactin…
LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery
Weikang Wan, Yifeng Zhu, Rutav Shah +1
We introduce LOTUS, a continual imitation learning algorithm that empowers a physical robot to continuously and efficiently learn to solve new manipulation tasks throughout its lif…
RoboHive: A Unified Framework for Robot Learning
Vikash Kumar, Rutav Shah, Gaoyue Zhou +5
We present RoboHive, a comprehensive software platform and ecosystem for research in the field of Robot Learning and Embodied Artificial Intelligence. Our platform encompasses a di…
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, thi…
MUTEX: Learning Unified Policies from Multimodal Task Specifications
Rutav Shah, Roberto Martín-Martín, Yuke Zhu
Humans use different modalities, such as speech, text, images, videos, etc., to communicate their intent and goals with teammates. For robots to become better assistants, we aim to…
RRL: Resnet as representation for Reinforcement Learning
Rutav Shah, Vikash Kumar
The ability to autonomously learn behaviors via direct interactions in uninstrumented environments can lead to generalist robots capable of enhancing productivity or providing care…