Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach

Abstract

We study the convergence to equilibrium of an underdamped Langevin equation that is controlled by a linear feedback force. Specifically, we are interested in sampling the possibly multimodal invariant probability distribution of a Langevin system at small noise (or low temperature), for which the dynamics can easily get trapped inside metastable subsets of the phase space. We follow [Chen et al., J. Math. Phys. 56, 113302, 2015] and consider a Langevin equation that is simulated at a high temperature, with the control playing the role of a friction that balances the additional noise so as to restore the original invariant measure at a lower temperature. We discuss different limits as the temperature ratio goes to infinity and prove convergence to a limit dynamics. It turns out that, depending on whether the lower ("target") or the higher ("simulation") temperature is fixed, the controlled dynamics converges either to the overdamped Langevin equation or to a deterministic gradient flow. This implies that (a) the ergodic limit and the large temperature separation limit do not commute in general, and that (b) it is not possible to accelerate the speed of convergence to the ergodic limit by making the temperature separation larger and larger. We discuss the implications of these observation from the perspective of stochastic optimisation algorithms and enhanced sampling schemes in molecular dynamics.

arXiv comment23 pages, 5 figures

References in corpus (21)

• Understanding deep learning requires rethinking generalization
• Qualitatively characterizing neural network optimization problems
• Rational Construction of Stochastic Numerical Methods for Molecular Sampling
• A consensus-based model for global optimization and its mean-field limit
• Sampling Can Be Faster Than Optimization
• Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion
• Accelerating diffusions
• Underdamped Langevin MCMC: A non-asymptotic analysis
• Irreversible Langevin samplers and variance reduction: a large deviation approach
• GENERIC formalism of a Vlasov-Fokker-Planck equation and connection to large-deviation principles
• Adaptive sampling of large deviations
• Data Assimilation: The Schrödinger Perspective
• Sharp entropy decay for hypocoercive and non-symmetric Fokker-Planck equations with linear drift
• Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations
• Quantification of coarse-graining error in Langevin and overdamped Langevin dynamics
• Mean-Field Langevin Dynamics and Energy Landscape of Neural Networks
• Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions
• CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing
• Mean-field Langevin System, Optimal Control and Deep Neural Networks
• Jarzynski's equality, fluctuation theorems, and variance reduction: Mathematical analysis and numerical algorithms
• Stochastic modified equations for the asynchronous stochastic gradient descent