One-Shot Visual Imitation Learning via Meta-Learning

Abstract

In order for a robot to be a generalist that can perform a wide range of jobs, it must be able to acquire a wide variety of skills quickly and efficiently in complex unstructured environments. High-capacity models such as deep neural networks can enable a robot to represent complex skills, but learning each skill from scratch then becomes infeasible. In this work, we present a meta-imitation learning method that enables a robot to learn how to learn more efficiently, allowing it to acquire new skills from just a single demonstration. Unlike prior methods for one-shot imitation, our method can scale to raw pixel inputs and requires data from significantly fewer prior tasks for effective learning of new skills. Our experiments on both simulated and real robot platforms demonstrate the ability to learn new tasks, end-to-end, from a single visual demonstration.

Conference on Robot Learning, 2017 (to appear). First two authors contributed equally. Video available at https://sites.google.com/view/one-shot-imitation

References in corpus (6)

• End to End Learning for Self-Driving Cars
• Theoretical Models of Learning to Learn
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• One-Shot Imitation Learning
• Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
• Learning Parameterized Skills

Cited by in corpus (81)

• How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
• An Algorithmic Perspective on Imitation Learning
• Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review
• Meta-Reinforcement Learning of Structured Exploration Strategies
• Evolved Policy Gradients
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills
• Sample Efficient Adaptive Text-to-Speech
• BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
• Generative Adversarial Imitation from Observation
• Learning to design from humans: Imitating human designers through deep learning
• Universal Planning Networks
• A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
• Unsupervised Meta-Learning for Reinforcement Learning
• Meta-Learning Requires Meta-Augmentation
• The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors
• Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning
• Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
• Few-Shot Goal Inference for Visuomotor Learning and Planning
• Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• Metalearned Neural Memory
• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
• Probabilistic Model-Agnostic Meta-Learning
• Learning Latent Plans from Play
• On-Policy Robot Imitation Learning from a Converging Supervisor
• Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
• RoboNet: Large-Scale Multi-Robot Learning
• Imitation Learning with Additional Constraints on Motion Style using Parametric Bias
• Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
• Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
• Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation
• Learning Navigation Subroutines from Egocentric Videos
• Recurrent Neural Network Control of a Hybrid Dynamic Transfemoral Prosthesis with EdgeDRNN Accelerator
• Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning
• Offline Meta-Reinforcement Learning with Advantage Weighting
• A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives
• OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research
• Learning Multi-Stage Tasks with One Demonstration via Self-Replay
• MetaPred: Meta-Learning for Clinical Risk Prediction with Limited Patient Electronic Health Records
• Vision-based Robot Manipulation Learning via Human Demonstrations
• Robust Constrained-MDPs: Soft-Constrained Robust Policy Optimization under Model Uncertainty
• Learning Alternative Ways of Performing a Task
• Formulating Camera-Adaptive Color Constancy as a Few-shot Meta-Learning Problem
• Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation
• Teaching Robots Novel Objects by Pointing at Them
• FOAL: Fast Online Adaptive Learning for Cardiac Motion Estimation
• Chameleon: Learning Model Initializations Across Tasks With Different Schemas
• Generalized Decision Transformer for Offline Hindsight Information Matching
• Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control
• Provable Representation Learning for Imitation Learning via Bi-level Optimization
• A Geometric Perspective on Visual Imitation Learning
• Semi-Supervised and Active Few-Shot Learning with Prototypical Networks
• A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes
• SAFARI: Safe and Active Robot Imitation Learning with Imagination
• ToolNet: Using Commonsense Generalization for Predicting Tool Use for Robot Plan Synthesis
• MELD: Meta-Reinforcement Learning from Images via Latent State Models
• Transferable Active Grasping and Real Embodied Dataset
• Lyapunov Robust Constrained-MDPs: Soft-Constrained Robustly Stable Policy Optimization under Model Uncertainty
• A Simple Approach to Continual Learning by Transferring Skill Parameters
• VIABLE: Fast Adaptation via Backpropagating Learned Loss
• Plan Arithmetic: Compositional Plan Vectors for Multi-Task Control
• PLOTS: Procedure Learning from Observations using Subtask Structure
• Knowledge Transfer Between Robots with Similar Dynamics for High-Accuracy Impromptu Trajectory Tracking
• SQUIRL: Robust and Efficient Learning from Video Demonstration of Long-Horizon Robotic Manipulation Tasks
• Few-Shot Bayesian Imitation Learning with Logical Program Policies
• Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning
• Extending Policy from One-Shot Learning through Coaching
• Shared Multi-Task Imitation Learning for Indoor Self-Navigation
• State Representation Learning from Demonstration
• Meta Adaptation using Importance Weighted Demonstrations
• When Autonomous Systems Meet Accuracy and Transferability through AI: A Survey
• Rearranging the Environment to Maximize Energy with a Robotic Circuit Drawing
• Neural Rate Control for Video Encoding using Imitation Learning
• Computational principles of intelligence: learning and reasoning with neural networks
• Learning Data-Driven Objectives to Optimize Interactive Systems
• Multi-Instance Aware Localization for End-to-End Imitation Learning
• Episodic Memory Model for Learning Robotic Manipulation Tasks
• Self-supervised Learning for Precise Pick-and-place without Object Model
• Autonomous Industrial Management via Reinforcement Learning: Self-Learning Agents for Decision-Making -- A Review
• On the potential for open-endedness in neural networks