Multi-Agent Generative Adversarial Imitation Learning

Abstract

Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-agent imitation learning for general Markov games, where we build upon a generalized notion of inverse reinforcement learning. We further introduce a practical multi-agent actor-critic algorithm with good empirical performance. Our method can be used to imitate complex behaviors in high-dimensional environments with multiple cooperative or competing agents.

Cited by in corpus (21)

• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
• Modeling Human Driving Behavior through Generative Adversarial Imitation Learning
• Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey
• Constructing Unrestricted Adversarial Examples with Generative Models
• From Screens to Scenes: A Survey of Embodied AI in Healthcare
• Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
• A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning
• Multi-task Maximum Entropy Inverse Reinforcement Learning
• Imitation Learning: Progress, Taxonomies and Challenges
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization
• TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations
• Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrations
• Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration
• RITA: Boost Driving Simulators with Realistic Interactive Traffic Flow
• Multi-Agent Interactions Modeling with Correlated Policies
• Dynamic Relational Inference in Multi-Agent Trajectories
• Learning Altruistic Behaviours in Reinforcement Learning without External Rewards
• DM${}^2$: Decentralized Multi-Agent Reinforcement Learning for Distribution Matching
• Imitation with Neural Density Models
• FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams
• Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts