Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games

Abstract

We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large $N$-player stochastic differential games. Following the idea of fictitious play, we recast the $N$-player game into $N$ decoupled decision problems (one for each player) and solve them iteratively. The individual decision problem is characterized by a semilinear Hamilton-Jacobi-Bellman equation, to solve which we employ the recently developed deep BSDE method. The resulted algorithm can solve large $N$-player games for which conventional numerical methods would suffer from the curse of dimensionality. Multiple numerical examples involving identical or heterogeneous agents, with risk-neutral or risk-sensitive objectives, are tested to validate the accuracy of the proposed algorithm in large group games. Even for a fifty-player game with the presence of common noise, the proposed algorithm still finds the approximate Nash equilibrium accurately, which, to our best knowledge, is difficult to achieve by other numerical algorithms.

References in corpus (12)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Solving high-dimensional partial differential equations using deep learning
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
• Convergence of the Deep BSDE Method for Coupled FBSDEs
• Uniformly Accurate Machine Learning Based Hydrodynamic Models for Kinetic Equations
• Universal approximations of permutation invariant/equivariant functions by deep neural networks
• Deep Fictitious Play for Stochastic Differential Games
• Numerical Probabilistic Approach to MFG

Cited by in corpus (9)

• Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning
• Convergence of the Deep BSDE Method for Coupled FBSDEs
• An overview on deep learning-based approximation methods for partial differential equations
• Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks
• KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning
• Convergence of Deep Fictitious Play for Stochastic Differential Games
• Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing
• Deep Fictitious Play for Stochastic Differential Games
• Large-Scale Multi-Agent Deep FBSDEs