On the Generalised Langevin Equation for Simulated Annealing
arXiv:2003.06448 · doi:10.1137/21M1462970
Abstract
In this paper, we consider the generalised (higher order) Langevin equation for the purpose of simulated annealing and optimisation of nonconvex functions. Our approach modifies the underdamped Langevin equation by replacing the Brownian noise with an appropriate Ornstein-Uhlenbeck process to account for memory in the system. Under reasonable conditions on the loss function and the annealing schedule, we establish convergence of the continuous time dynamics to a global minimum. In addition, we investigate the performance numerically and show better performance and higher exploration of the state space compared to the underdamped and overdamped Langevin dynamics with the same annealing schedule.
arXiv comment37 pages, 4 figures
References in corpus (17)
- An overview of gradient descent optimization algorithms
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
- Langevin equation with colored noise for constant-temperature molecular dynamics simulations
- Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
- Data-driven parameterization of the generalized Langevin equation
- A consensus-based model for global optimization and its mean-field limit
- Non-Equilibrium Statistical Mechanics of Strongly Anharmonic Chains of Oscillators
- Sampling Can Be Faster Than Optimization
- Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion
- Variance Reduction using Nonreversible Langevin Samplers
- Underdamped Langevin MCMC: A non-asymptotic analysis
- Poincaré and logarithmic Sobolev inequalities by decomposition of the energy landscape
- High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm
- Breaking Reversibility Accelerates Langevin Dynamics for Global Non-Convex Optimization
- A consensus-based global optimization method for high dimensional machine learning problems
- Evaluating functions of positive-definite matrices using colored noise thermostats
- Generalized calculus and application to interacting particles on a graph