Relational Mimic for Visual Adversarial Imitation Learning

Abstract

In this work, we introduce a new method for imitation learning from video demonstrations. Our method, Relational Mimic (RM), improves on previous visual imitation learning methods by combining generative adversarial networks and relational learning. RM is flexible and can be used in conjunction with other recent advances in generative adversarial imitation learning to better address the need for more robust and sample-efficient approaches. In addition, we introduce a new neural network architecture that improves upon the previous state-of-the-art in reinforcement learning and illustrate how increasing the relational reasoning capabilities of the agent enables the latter to achieve increasingly higher performance in a challenging locomotion task with pixel inputs. Finally, we study the effects and contributions of relational learning in policy evaluation, policy improvement and reward learning through ablation studies.

References in corpus (19)

• Deep Residual Learning for Image Recognition
• End to End Learning for Self-Driving Cars
• Self-Attention Generative Adversarial Networks
• Dueling Network Architectures for Deep Reinforcement Learning
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• A simple neural network module for relational reasoning
• Deep Reinforcement Learning that Matters
• Stand-Alone Self-Attention in Vision Models
• Quantifying Generalization in Reinforcement Learning
• Learning human behaviors from motion capture by adversarial imitation
• Synthesizing Programs for Images using Reinforced Adversarial Learning
• Scalable agent alignment via reward modeling: a research direction
• AI Safety Gridworlds
• Generative Adversarial Imitation from Observation
• Are GANs Created Equal? A Large-Scale Study
• Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
• Inverse Reward Design
• Model-Free Imitation Learning with Policy Optimization
• Risk-Sensitive Generative Adversarial Imitation Learning