A Geometric Perspective on Visual Imitation Learning
arXiv:2003.02768
Abstract
We consider the problem of visual imitation learning without human supervision (e.g. kinesthetic teaching or teleoperation), nor access to an interactive reinforcement learning (RL) training environment. We present a geometric perspective to derive solutions to this problem. Specifically, we propose VGS-IL (Visual Geometric Skill Imitation Learning), an end-to-end geometry-parameterized task concept inference method, to infer globally consistent geometric feature association rules from human demonstration video frames. We show that, instead of learning actions from image pixels, learning a geometry-parameterized task concept provides an explainable and invariant representation across demonstrator to imitator under various environmental settings. Moreover, such a task concept representation provides a direct link with geometric vision based controllers (e.g. visual servoing), allowing for efficient mapping of high-level task concepts to low-level robot actions.
submitted to IROS 2020
References in corpus (8)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Neural Message Passing for Quantum Chemistry
- Generative Adversarial Imitation Learning
- One-Shot Visual Imitation Learning via Meta-Learning
- Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller
- Graph-Structured Visual Imitation
- Robot eye-hand coordination learning by watching human demonstrations: a task function approximation approach
- KETO: Learning Keypoint Representations for Tool Manipulation