8 papers
Large-scale empirical tuning and comparison of default optimizers for variational inference
Trevor Campbell, Jonathan H. Huggins, Kyurae Kim +1
Black-box variational inference (BBVI) is a methodology for posterior approximation that relies on stochastic optimization. In practice, the stochastic optimizers underpinning BBVI…
Hessian-informed, Coordinate Friendly Hamiltonian Monte Carlo in Linear Time
Son Luu, Nikola Surjanovic, Zuheng Xu +2
Riemannian Hamiltonian Monte Carlo (RHMC) is a promising MCMC methodology thanks to its ability to accommodate position-dependent preconditioning and multi-step proposals. While RH…
Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space
Kyurae Kim, Qiang Fu, Yi-An Ma +2
For approximating a target distribution given only its unnormalized log-density, stochastic gradient-based variational inference (VI) algorithms are a popular approach. For example…
AutoGD: Automatic Learning Rate Selection for Gradient Descent
Nikola Surjanovic, Alexandre Bouchard-Côté, Trevor Campbell
The performance of gradient-based optimization methods, such as standard gradient descent (GD), greatly depends on the choice of learning rate. However, it can require a non-trivia…
AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
Nikola Surjanovic, Alexandre Bouchard-Côté, Trevor Campbell
The learning rate is an important tuning parameter for stochastic gradient descent (SGD) and can greatly influence its performance. However, appropriate selection of a learning rat…
AutoStep: Locally adaptive involutive MCMC
Tiange Liu, Nikola Surjanovic, Miguel Biron-Lattes +2
Many common Markov chain Monte Carlo (MCMC) kernels can be formulated using a deterministic involutive proposal with a step size parameter. Selecting an appropriate step size is of…