6 papers
Adaptive Momentum and Nonlinear Damping for Neural Network Training
Aikaterini Karoni, Rajit Rajpal, Benedict Leimkuhler +1
Momentum Stochastic Gradient Descent (mSGD) relies on a fixed momentum coefficient shared across all parameters, failing to account for the heterogeneous structure of modern loss l…
Accelerated sampling using SamAdams variable timesteps and position-adaptive Langevin dynamics
Benedict Leimkuhler, Peter A. Whalley
We introduce an accelerated Langevin-based sampling method that is based on two complementary devices: \emph{SamAdams} adaptive timestepping, which automatically shrinks the effect…
Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks
Rajit Rajpal, Benedict Leimkuhler, Yuanhao Jiang
Bayesian neural networks (BNNs) require scalable sampling algorithms to approximate posterior distributions over parameters. Existing stochastic gradient Markov Chain Monte Carlo (…
Numerical integrators for confined Langevin dynamics
B. Leimkuhler, A. Sharma, M. V. Tretyakov
We derive and analyze numerical methods for underdamped (kinetic) Langevin dynamics in a domain with elastic reflection at the boundary. First-order approximations are based on an…
A Langevin sampling algorithm inspired by the Adam optimizer
Benedict Leimkuhler, René Lohmann, Peter Whalley
We present a framework for adaptive-stepsize MCMC sampling based on time-rescaled Langevin dynamics, in which the stepsize variation is dynamically driven by an additional degree o…
Efficient Langevin sampling with position-dependent diffusion
Eugen Bronasco, Benedict Leimkuhler, Dominic Phillips +1
We introduce a numerical method for Brownian dynamics with position dependent diffusion tensor which is second order accurate for sampling the invariant measure while requiring onl…