Transformers for One-Shot Visual Imitation

Abstract

Humans are able to seamlessly visually imitate others, by inferring their intentions and using past experience to achieve the same end goal. In other words, we can parse complex semantic knowledge from raw video and efficiently translate that into concrete motor control. Is it possible to give a robot this same capability? Prior research in robot imitation learning has created agents which can acquire diverse skills from expert human operators. However, expanding these techniques to work with a single positive example during test time is still an open challenge. Apart from control, the difficulty stems from mismatches between the demonstrator and robot domains. For example, objects may be placed in different locations (e.g. kitchen layouts are different in every house). Additionally, the demonstration may come from an agent with different morphology and physical appearance (e.g. human), so one-to-one action correspondences are not available. This paper investigates techniques which allow robots to partially bridge these domain gaps, using their past experience. A neural network is trained to mimic ground truth robot actions given context video from another agent, and must generalize to unseen task instances when prompted with new videos during test time. We hypothesize that our policy representations must be both context driven and dynamics aware in order to perform these tasks. These assumptions are baked into the neural network using the Transformers attention mechanism and a self-supervised inverse dynamics loss. Finally, we experimentally determine that our method accomplishes a $\sim 2$x improvement in terms of task success rate over prior baselines in a suite of one-shot manipulation tasks.

For code and project video please check our website: https://oneshotfeatures.github.io/

References in corpus (18)

• Sequence to Sequence Learning with Neural Networks
• A Simple Framework for Contrastive Learning of Visual Representations
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Two-Stream Convolutional Networks for Action Recognition in Videos
• Bootstrap your own latent: A new approach to self-supervised Learning
• End to End Learning for Self-Driving Cars
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• Reinforcement Learning with Augmented Data
• Long Short-Term Memory-Networks for Machine Reading
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning
• Self-Supervised Policy Adaptation during Deployment
• RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
• Goal-conditioned Imitation Learning
• Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement
• Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller
• From virtual demonstration to real-world manipulation using LSTM and MDN
• Watch, Try, Learn: Meta-Learning from Demonstrations and Reward

Cited by in corpus (3)

• Decision Transformer: Reinforcement Learning via Sequence Modeling
• BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning
• Manipulator-Independent Representations for Visual Imitation