A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems

Abstract

We propose a PDE-based accelerated gradient algorithm for optimal feedback controls of McKean-Vlasov dynamics that involve mean-field interactions both in the state and action. The method exploits a forward-backward splitting approach and iteratively refines the approximate controls based on the gradients of smooth costs, the proximal maps of nonsmooth costs, and dynamically updated momentum parameters. At each step, the state dynamics is approximated via a particle system, and the required gradient is evaluated through a coupled system of nonlocal linear PDEs. The latter is solved by finite difference approximation or neural network-based residual approximation, depending on the state dimension. We present exhaustive numerical experiments for low and high-dimensional mean-field control problems, including sparse stabilization of stochastic Cucker-Smale models, which reveal that our algorithm captures important structures of the optimal feedback control and achieves a robust performance with respect to parameter perturbation.

Accepted for publication by SIAM Journal on Scientific Computing

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• DGM: A deep learning algorithm for solving partial differential equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Semi-Lagrangian schemes for linear and fully non-linear diffusion equations
• Optimal consensus control of the Cucker-Smale model
• First order convergence of Milstein schemes for McKean-Vlasov equations and interacting particle systems
• A neural network based policy iteration algorithm with global $H^2$-superlinear convergence for stochastic games on domains
• Numerical Methods for Mean-Field-Type Optimal Control Problems
• A posteriori error estimates for fully coupled McKean-Vlasov forward-backward SDEs
• Linear-quadratic McKean-Vlasov stochastic control problems with random coefficients on finite and infinite horizon, and applications
• Optimal regularity of extended mean field controls and their piecewise constant approximation
• Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls

Cited by in corpus (1)

• Actor-Critic learning for mean-field control in continuous time