100 citations · 249 across the 16 of their papers we have counts for
16 papers
EXIMO: VLM Guided Exploration of VLA Policies
Bhavya Sukhija, Oliver Groth, Mohit Shridhar +5
How to efficiently finetune robot policies to learn new tasks on the fly? State of the art robotic manipulation policies are based on behaviour cloning of large vision-language-act…
What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agents
Jiaheng Hu, Mohit Shridhar, Caden Lu +4
Hierarchical vision-language-action (Hi-VLA) systems have emerged as a promising paradigm for complex robot manipulation, by using high-level VLM planners to decompose tasks into l…
ExoStart: Efficient learning for dexterous manipulation with sensorized exoskeleton demonstrations
Zilin Si, Jose Enrique Chen, M. Emre Karagozler +15
Recent advancements in teleoperation systems have enabled high-quality data collection for robotic manipulators, showing impressive results in learning manipulation at scale. This…
PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks
Markus Grotz, Mohit Shridhar, Tamim Asfour +1
Bimanual manipulation is challenging due to precise spatial and temporal coordination required between two arms. While there exist several real-world bimanual systems, there is a l…
Generative Image as Action Models
Mohit Shridhar, Yat Long Lo, Stephen James
Image-generation diffusion models have been fine-tuned to unlock new capabilities such as image-editing and novel view synthesis. Can we similarly unlock image-generation models fo…
Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation
Xiangtong Yao, Hongkuan Zhou, Oier Mees +12
Language-conditioned robot manipulation is an emerging field aimed at enabling seamless communication and cooperation between humans and robotic agents by teaching robots to compre…
GenSim: Generating Robotic Simulation Tasks via Large Language Models
Lirui Wang, Yiyang Ling, Zhecheng Yuan +6
Collecting large amounts of real-world interaction data to train general robotic policies is often prohibitively expensive, thus motivating the use of simulation data. However, exi…
AR2-D2:Training a Robot Without a Robot
Jiafei Duan, Yi Ru Wang, Mohit Shridhar +2
Diligently gathered human demonstrations serve as the unsung heroes empowering the progression of robot learning. Today, demonstrations are collected by training people to use spec…
Retrospectives on the Embodied AI Workshop
Matt Deitke, Dhruv Batra, Yonatan Bisk +36
We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped i…
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation
Mohit Shridhar, Lucas Manuelli, Dieter Fox
Transformers have revolutionized vision and natural language processing with their ability to scale with large datasets. But in robotic manipulation, data is both limited and expen…
CLIPort: What and Where Pathways for Robotic Manipulation
Mohit Shridhar, Lucas Manuelli, Dieter Fox
How can we imbue robots with the ability to manipulate objects precisely but also to reason about them in terms of abstract concepts? Recent works in manipulation have shown that e…
Language Grounding with 3D Objects
Jesse Thomason, Mohit Shridhar, Yonatan Bisk +2
Seemingly simple natural language requests to a robot are generally underspecified, for example "Can you bring me the wireless mouse?" Flat images of candidate mice may not provide…
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté +3
Given a simple request like Put a washed apple in the kitchen fridge, humans can reason in purely abstract terms by imagining action sequences and scoring their likelihood of succe…
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
Mohit Shridhar, Jesse Thomason, Daniel Gordon +5
We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to seque…
Interactive Visual Grounding of Referring Expressions for Human-Robot Interaction
Mohit Shridhar, David Hsu
This paper presents INGRESS, a robot system that follows human natural language instructions to pick and place everyday objects. The core issue here is the grounding of referring e…
Grounding Spatio-Semantic Referring Expressions for Human-Robot Interaction
Mohit Shridhar, David Hsu
The human language is one of the most natural interfaces for humans to interact with robots. This paper presents a robot system that retrieves everyday objects with unconstrained n…