One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks

Abstract

We consider the problem of learning multi-stage vision-based tasks on a real robot from a single video of a human performing the task, while leveraging demonstration data of subtasks with other objects. This problem presents a number of major challenges. Video demonstrations without teleoperation are easy for humans to provide, but do not provide any direct supervision. Learning policies from raw pixels enables full generality but calls for large function approximators with many parameters to be learned. Finally, compound tasks can require impractical amounts of demonstration data, when treated as a monolithic skill. To address these challenges, we propose a method that learns both how to learn primitive behaviors from video demonstrations and how to dynamically compose these behaviors to perform multi-stage tasks by "watching" a human demonstrator. Our results on a simulated Sawyer robot and real PR2 robot illustrate our method for learning a variety of order fulfillment and kitchen serving tasks with novel objects and raw pixel inputs.

arXiv commentVideo results available at https://sites.google.com/view/one-shot-hil

References in corpus (4)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Searching for Activation Functions
• TACO: Learning Task Decomposition via Temporal Alignment for Control
• Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets

Cited by in corpus (22)

• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
• Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
• Causal Navigation by Continuous-time Neural Networks
• A Behavioral Approach to Visual Navigation with Graph Localization Networks
• Imitation Learning for High Precision Peg-in-Hole Tasks
• Deep Imitation Learning for Bimanual Robotic Manipulation
• Adversarial Option-Aware Hierarchical Imitation Learning
• Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks
• To Follow or not to Follow: Selective Imitation Learning from Observations
• Learning Multi-Stage Tasks with One Demonstration via Self-Replay
• SORNet: Spatial Object-Centric Representations for Sequential Manipulation
• O2A: One-shot Observational learning with Action vectors
• Transferable Active Grasping and Real Embodied Dataset
• Online Baum-Welch algorithm for Hierarchical Imitation Learning
• Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems with Heterogeneous Sensor Data
• SQUIRL: Robust and Efficient Learning from Video Demonstration of Long-Horizon Robotic Manipulation Tasks
• Transferable Task Execution from Pixels through Deep Planning Domain Learning
• Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
• Meta Adaptation using Importance Weighted Demonstrations
• Learning from Interventions using Hierarchical Policies for Safe Learning