Multi-agent Inverse Reinforcement Learning for Two-person Zero-sum Games

Abstract

The focus of this paper is a Bayesian framework for solving a class of problems termed multi-agent inverse reinforcement learning (MIRL). Compared to the well-known inverse reinforcement learning (IRL) problem, MIRL is formalized in the context of stochastic games, which generalize Markov decision processes to game theoretic scenarios. We establish a theoretical foundation for competitive two-agent zero-sum MIRL problems and propose a Bayesian solution approach in which the generative model is based on an assumption that the two agents follow a minimax bi-policy. Numerical results are presented comparing the Bayesian MIRL method with two existing methods in the context of an abstract soccer game. Investigation centers on relationships between the extent of prior information and the quality of learned rewards. Results suggest that covariance structure is more important than mean value in reward priors.

References in corpus (2)

• Move Evaluation in Go Using Deep Convolutional Neural Networks
• A Multiagent Reinforcement Learning Algorithm with Non-linear Dynamics

Cited by in corpus (11)

• Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games
• MADRaS : Multi Agent Driving Simulator
• Fairness-aware Competitive Bidding Influence Maximization in Social Networks
• Inverse Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning
• Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrations
• Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration
• Off-Policy Exploitability-Evaluation in Two-Player Zero-Sum Markov Games
• Using Multi-Agent Reinforcement Learning in Auction Simulations
• Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts
• Cooperative Multi-Agent Policy Gradients with Sub-optimal Demonstration
• Equilibrium Inverse Reinforcement Learning for Ride-hailing Vehicle Network