One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning

Abstract

Humans and animals are capable of learning a new behavior by observing others perform the skill just once. We consider the problem of allowing a robot to do the same -- learning from a raw video pixels of a human, even when there is substantial domain shift in the perspective, environment, and embodiment between the robot and the observed human. Prior approaches to this problem have hand-specified how human and robot actions correspond and often relied on explicit human pose detection systems. In this work, we present an approach for one-shot learning from a video of a human by using human and robot demonstration data from a variety of previous tasks to build up prior knowledge through meta-learning. Then, combining this prior knowledge and only a single video demonstration from a human, the robot can perform the task that the human demonstrated. We show experiments on both a PR2 arm and a Sawyer arm, demonstrating that after meta-learning, the robot can learn to place, push, and pick-and-place new objects using just one video of a human performing the manipulation.

arXiv commentFirst two authors contributed equally. Video available at https://sites.google.com/view/daml

References in corpus (16)

• WaveNet: A Generative Model for Raw Audio
• Deep Domain Confusion: Maximizing for Domain Invariance
• Theoretical Models of Learning to Learn
• One-Shot Imitation Learning
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Virtual to Real Reinforcement Learning for Autonomous Driving
• Third-Person Imitation Learning
• PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
• Learning to Generalize: Meta-Learning for Domain Generalization
• Learning a visuomotor controller for real world robotic grasping using simulated depth images
• Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
• Time-Contrastive Networks: Self-Supervised Learning from Video
• Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
• Learning Robot Activities from First-Person Human Videos Using Convolutional Future Regression
• What Would You Do? Acting by Learning to Predict
• Satellite Image-based Localization via Learned Embeddings

Cited by in corpus (48)

• Evolved Policy Gradients
• Sample Efficient Adaptive Text-to-Speech
• Frustratingly Simple Domain Generalization via Image Stylization
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale
• Few-Shot Goal Inference for Visuomotor Learning and Planning
• Modular meta-learning
• Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations
• Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors
• V2CNet: A Deep Learning Framework to Translate Videos to Commands for Robotic Manipulation
• On-Policy Robot Imitation Learning from a Converging Supervisor
• Robot eye-hand coordination learning by watching human demonstrations: a task function approximation approach
• Imitation Learning for High Precision Peg-in-Hole Tasks
• Towards Cross-Granularity Few-Shot Learning: Coarse-to-Fine Pseudo-Labeling with Visual-Semantic Meta-Embedding
• Learning Navigation Subroutines from Egocentric Videos
• Learning Actionable Representations from Visual Observations
• Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity
• From explanation to synthesis: Compositional program induction for learning from demonstration
• One-Shot Object Localization Using Learnt Visual Cues via Siamese Networks
• Learning Multi-Stage Tasks with One Demonstration via Self-Replay
• Long-term Human Motion Prediction with Scene Context
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback
• Transformers for One-Shot Visual Imitation
• Vision-based Robot Manipulation Learning via Human Demonstrations
• Meta-Learning Multi-task Communication
• Hindsight Generative Adversarial Imitation Learning
• Noether Networks: Meta-Learning Useful Conserved Quantities
• Teaching Robots Novel Objects by Pointing at Them
• Reinforcement Learning with Videos: Combining Offline Observations with Interaction
• Data-efficient visuomotor policy training using reinforcement learning and generative models
• Federated Imitation Learning: A Privacy Considered Imitation Learning Framework for Cloud Robotic Systems with Heterogeneous Sensor Data
• SAFARI: Safe and Active Robot Imitation Learning with Imagination
• Shaping embodied agent behavior with activity-context priors from egocentric video
• Cross-Domain Imitation Learning with a Dual Structure
• MELD: Meta-Reinforcement Learning from Images via Latent State Models
• Hybrid Reinforcement Learning with Expert State Sequences
• Visual Task Progress Estimation with Appearance Invariant Embeddings for Robot Control and Planning
• Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learning
• Evaluation of state representation methods in robot hand-eye coordination learning from demonstration
• Invariant Feature Learning for Sensor-based Human Activity Recognition
• AGENT: A Benchmark for Core Psychological Reasoning
• Hybrid system identification using switching density networks
• PLOTS: Procedure Learning from Observations using Subtask Structure
• Manipulator-Independent Representations for Visual Imitation
• A Markov Decision Process Approach to Active Meta Learning
• Agent Probing Interaction Policies
• Trajectory-based Learning for Ball-in-Maze Games
• Phase Portraits as Movement Primitives for Fast Humanoid Robot Control
• Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain