6 papers
Hessian-informed, Coordinate Friendly Hamiltonian Monte Carlo in Linear Time
Son Luu, Nikola Surjanovic, Zuheng Xu +2
Riemannian Hamiltonian Monte Carlo (RHMC) is a promising MCMC methodology thanks to its ability to accommodate position-dependent preconditioning and multi-step proposals. While RH…
AutoGD: Automatic Learning Rate Selection for Gradient Descent
Nikola Surjanovic, Alexandre Bouchard-Côté, Trevor Campbell
The performance of gradient-based optimization methods, such as standard gradient descent (GD), greatly depends on the choice of learning rate. However, it can require a non-trivia…
AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
Nikola Surjanovic, Alexandre Bouchard-Côté, Trevor Campbell
The learning rate is an important tuning parameter for stochastic gradient descent (SGD) and can greatly influence its performance. However, appropriate selection of a learning rat…
AutoStep: Locally adaptive involutive MCMC
Tiange Liu, Nikola Surjanovic, Miguel Biron-Lattes +2
Many common Markov chain Monte Carlo (MCMC) kernels can be formulated using a deterministic involutive proposal with a step size parameter. Selecting an appropriate step size is of…
Is Gibbs sampling faster than Hamiltonian Monte Carlo on GLMs?
Son Luu, Zuheng Xu, Nikola Surjanovic +3
The Hamiltonian Monte Carlo (HMC) algorithm is often lauded for its ability to effectively sample from high-dimensional distributions. In this paper we challenge the presumed domin…
Uniform Ergodicity of Parallel Tempering With Efficient Local Exploration
Nikola Surjanovic, Saifuddin Syed, Alexandre Bouchard-Côté +1
Non-reversible parallel tempering (NRPT) is an effective algorithm for sampling from target distributions with complex geometry, such as those arising from posterior distributions…