InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
arXiv:1703.08840
Abstract
The goal of imitation learning is to mimic expert behavior without access to an explicit reward signal. Expert demonstrations provided by humans, however, often show significant variability due to latent factors that are typically not explicitly modeled. In this paper, we propose a new algorithm that can infer the latent structure of expert demonstrations in an unsupervised way. Our method, built on top of Generative Adversarial Imitation Learning, can not only imitate complex behaviors, but also learn interpretable and meaningful representations of complex behavioral data, including visual demonstrations. In the driving domain, we show that a model learned from human demonstrations is able to both accurately reproduce a variety of behaviors and accurately anticipate human actions using raw visual inputs. Compared with various baselines, our method can better capture the latent structure underlying expert demonstrations, often recovering semantically meaningful factors of variation in the data.
14 pages, NIPS 2017
References in corpus (2)
Cited by in corpus (40)
- Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives
- Deep Reinforcement Learning: An Overview
- A Survey of Deep RL and IL for Autonomous Driving Policy Learning
- TrajGAIL: Generating Urban Vehicle Trajectories using Generative Adversarial Imitation Learning
- Reinforcement and Imitation Learning for Diverse Visuomotor Skills
- Universal Planning Networks
- Active Learning in Robotics: A Review of Control Principles
- A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
- Robust Imitation of Diverse Behaviors
- Generating Multi-Agent Trajectories using Programmatic Weak Supervision
- SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
- Generative Adversarial Self-Imitation Learning
- Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
- MADRaS : Multi Agent Driving Simulator
- Rethinking Closed-loop Training for Autonomous Driving
- Learning Temporal Strategic Relationships using Generative Adversarial Imitation Learning
- Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
- Imitation Learning with Additional Constraints on Motion Style using Parametric Bias
- Multi-task Maximum Entropy Inverse Reinforcement Learning
- Recurrent Neural Network Control of a Hybrid Dynamic Transfemoral Prosthesis with EdgeDRNN Accelerator
- Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation
- Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
- Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
- Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrations
- Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
- Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation
- Learning Multi-Task Transferable Rewards via Variational Inverse Reinforcement Learning
- RITA: Boost Driving Simulators with Realistic Interactive Traffic Flow
- Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion
- Multi-Level Sequence GAN for Group Activity Recognition
- CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving
- Adversarial Constraint Learning for Structured Prediction
- LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning
- Toward Imitating Visual Attention of Experts in Software Development Tasks
- Active Imitation Learning from Multiple Non-Deterministic Teachers: Formulation, Challenges, and Algorithms
- Learning without Knowing: Unobserved Context in Continuous Transfer Reinforcement Learning
- Theory of Machine Networks: A Case Study
- Hierarchical Policies for Cluttered-Scene Grasping with Latent Plans
- Mature GAIL: Imitation Learning for Low-level and High-dimensional Input using Global Encoder and Cost Transformation
- PODNet: A Neural Network for Discovery of Plannable Options