8 papers
Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering
Zikui Cai, Kaushal Janga, Tan Dat Dao +15
Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. Ho…
Memory Retrieval in Visuomotor Policies for Long-Horizon Robot Control
Rutav Shah, Yisu Li, Femi Bello +2
General-purpose robots operating in partially observable environments, such as homes, require memory to support autonomy. They must recall diverse information from the past, such a…
Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks
Rutav Shah, Rajat Kumar Jenamani, Xiaohan Zhang +5
Many robotic tasks require short-term memory, whether it's retrieving an object that's no longer visible or turning off an appliance after a set period. Yet, most visuomotor polici…
Searching in Space and Time: Unified Memory-Action Loops for Open-World Object Retrieval
Taijing Chen, Sateesh Kumar, Junhong Xu +3
Service robots must retrieve objects in dynamic, open-world settings where requests may reference attributes ("the red mug"), spatial context ("the mug on the table"), or past stat…
GET-USE: Learning Generalized Tool Usage for Bimanual Mobile Manipulation via Simulated Embodiment Extensions
Bohan Wu, Paul de La Sayette, Li Fei-Fei +1
The ability to use random objects as tools in a generalizable manner is a missing piece in robots' intelligence today to boost their versatility and problem-solving capabilities. S…
MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation
Chengshu Li, Mengdi Xu, Arpit Bahety +11
Imitation learning from large-scale, diverse human demonstrations has been shown to be effective for training robots, but collecting such data is costly and time-consuming. This ch…
MimicDroid: In-Context Learning for Humanoid Robot Manipulation from Human Play Videos
Rutav Shah, Shuijing Liu, Qi Wang +5
We aim to enable humanoid robots to efficiently solve new manipulation tasks from a few video examples. In-context learning (ICL) is a promising framework for achieving this goal d…
RoboSSM: Scalable In-context Imitation Learning via State-Space Models
Youngju Yoo, Jiaheng Hu, Yifeng Zhu +4
In-context imitation learning (ICIL) enables robots to learn tasks from prompts consisting of just a handful of demonstrations. By eliminating the need for parameter updates at dep…