4 papers
AutoGD: Automatic Learning Rate Selection for Gradient Descent
Nikola Surjanovic, Alexandre Bouchard-Côté, Trevor Campbell
The performance of gradient-based optimization methods, such as standard gradient descent (GD), greatly depends on the choice of learning rate. However, it can require a non-trivia…
AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
Nikola Surjanovic, Alexandre Bouchard-Côté, Trevor Campbell
The learning rate is an important tuning parameter for stochastic gradient descent (SGD) and can greatly influence its performance. However, appropriate selection of a learning rat…
AutoStep: Locally adaptive involutive MCMC
Tiange Liu, Nikola Surjanovic, Miguel Biron-Lattes +2
Many common Markov chain Monte Carlo (MCMC) kernels can be formulated using a deterministic involutive proposal with a step size parameter. Selecting an appropriate step size is of…
Is Gibbs sampling faster than Hamiltonian Monte Carlo on GLMs?
Son Luu, Zuheng Xu, Nikola Surjanovic +3
The Hamiltonian Monte Carlo (HMC) algorithm is often lauded for its ability to effectively sample from high-dimensional distributions. In this paper we challenge the presumed domin…