Uniform-in-time propagation of chaos for kinetic mean field Langevin dynamics

Abstract

We study the kinetic mean field Langevin dynamics under the functional convexity assumption of the mean field energy functional. Using hypocoercivity, we first establish the exponential convergence of the mean field dynamics and then show the corresponding $N$-particle system converges exponentially in a rate uniform in $N$ modulo a small error. Finally we study the short-time regularization effects of the dynamics and prove its uniform-in-time propagation of chaos property in both the Wasserstein and entropic sense. Our results can be applied to the training of two-layer neural networks with momentum and we include the numerical experiments.

arXiv comment54 pages, 5 figures, 1 table; errors in the previous version were corrected; accepted version

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• A Mean Field View of the Landscape of Two-Layers Neural Networks
• Propagation of chaos: a review of models, methods and applications. II. Applications
• Trainability and Accuracy of Neural Networks: An Interacting Particle System Approach
• Uniform long-time and propagation of chaos estimates for mean field kinetic particles in non-convex landscapes
• An Improved Analysis of Stochastic Gradient Descent with Momentum
• Continuous Time Analysis of Momentum Methods
• Almost sure convergence rates for Stochastic Gradient Descent and Stochastic Heavy Ball
• Ergodicity of the underdamped mean-field Langevin dynamics
• Sharp uniform-in-time mean-field convergence for singular periodic Riesz flows

Cited by in corpus (3)

• Uniform-in-time propagation of chaos for mean field Langevin dynamics
• Size of chaos for Gibbs measures of mean field interacting diffusions
• Self-interacting approximation to McKean-Vlasov long-time limit: a Markov chain Monte Carlo method