Learning Mean-Field Games
arXiv:1901.09585
Abstract
This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and explains that naively combining Q-learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes a Q-learning algorithm with Boltzmann policy (GMF-Q), with analysis of convergence property and computational complexity. The experiments on repeated Ad auction problems demonstrate that this GMF-Q algorithm is efficient and robust in terms of convergence and learning accuracy. Moreover, its performance is superior in convergence, stability, and learning ability, when compared with existing algorithms for multi-agent reinforcement learning.
Published in NeurIPS 2019
References in corpus (7)
- Mean Field Multi-Agent Reinforcement Learning
- On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning
- Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
- Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization
- Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches
- A Mean Field Game of Portfolio Trading and Its Consequences On Perceived Correlations
- Mean Field Stochastic Games with Binary Action Spaces and Monotone Costs