30 citations · 30 across the 6 of their papers we have counts for
8 papers
Low-Rank Friction for Memory-Efficient Transformer Pretraining
Rajit Rajpal, Benedict Leimkuhler
iKFAD is a recently proposed optimiser that replaces adaptive learning rates with adaptive friction in the momentum dynamics, yet performs as well as Adam. Its limitation is that t…
Accelerated sampling using SamAdams variable timesteps and position-adaptive Langevin dynamics
Benedict Leimkuhler, Peter A. Whalley
We introduce an accelerated Langevin-based sampling method that is based on two complementary devices: \emph{SamAdams} adaptive timestepping, which automatically shrinks the effect…
Adaptive Momentum and Nonlinear Damping for Neural Network Training
Aikaterini Karoni, Rajit Rajpal, Benedict Leimkuhler +1
Momentum Stochastic Gradient Descent (mSGD) relies on a fixed momentum coefficient shared across all parameters, failing to account for the heterogeneous structure of modern loss l…
Adaptive Stepsizing for Stochastic Gradient Langevin Dynamics in Bayesian Neural Networks
Rajit Rajpal, Benedict Leimkuhler, Yuanhao Jiang
Bayesian neural networks (BNNs) require scalable sampling algorithms to approximate posterior distributions over parameters. Existing stochastic gradient Markov Chain Monte Carlo (…
A Langevin sampling algorithm inspired by the Adam optimizer
Benedict Leimkuhler, René Lohmann, Peter Whalley
We present a framework for adaptive-stepsize MCMC sampling based on time-rescaled Langevin dynamics, in which the stepsize variation is dynamically driven by an additional degree o…
Efficient Langevin sampling with position-dependent diffusion
Eugen Bronasco, Benedict Leimkuhler, Dominic Phillips +1
We introduce a numerical method for Brownian dynamics with position dependent diffusion tensor which is second order accurate for sampling the invariant measure while requiring onl…
Adaptive stepsize algorithms for Langevin dynamics
Alix Leroy, Benedict Leimkuhler, Jonas Latz +1
We discuss the design of an invariant measure-preserving transformed dynamics for the numerical treatment of Langevin dynamics based on rescaling of time, with the goal of sampling…
Local and Global Perspectives on Diffusion Maps in the Analysis of Molecular Systems
Zofia Trstanova, Ben Leimkuhler, Tony Lelièvre
Diffusion maps approximate the generator of Langevin dynamics from simulation data. They afford a means of identifying the slowly-evolving principal modes of high-dimensional molec…