BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning

Abstract

In this paper, we study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks, a long-standing challenge in robot learning. We approach the challenge from an imitation learning perspective, aiming to study how scaling and broadening the data collected can facilitate such generalization. To that end, we develop an interactive and flexible imitation learning system that can learn from both demonstrations and interventions and can be conditioned on different forms of information that convey the task, including pre-trained embeddings of natural language or videos of humans performing the task. When scaling data collection on a real robot to more than 100 distinct tasks, we find that this system can perform 24 unseen manipulation tasks with an average success rate of 44%, without any robot demonstrations for those tasks.

arXiv commentCoRL 2021, 23 pages

References in corpus (5)

• One-Shot Visual Imitation Learning via Meta-Learning
• DART: Noise Injection for Robust Imitation Learning
• Active Domain Randomization
• Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
• Transformers for One-Shot Visual Imitation

Cited by in corpus (6)

• Text2Motion: From Natural Language Instructions to Feasible Plans
• "No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy
• PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation
• Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning
• Interactive Task Encoding System for Learning-from-Observation
• Learning Top-k Subtask Planning Tree based on Discriminative Representation Pre-training for Decision Making