105 citations · 265 across the 6 of their papers we have counts for
5 papers · 1 filter
Stochastic Gradient Descent as Approximate Bayesian Inference
Stephan Mandt, Matthew D. Hoffman, David M. Blei
Stochastic Gradient Descent with a constant learning rate (constant SGD) simulates a Markov chain with a stationary distribution. With this perspective, we derive several new resul…
Exponential Family Embeddings
Maja R. Rudolph, Francisco J. R. Ruiz, Stephan Mandt +1
Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. In this paper, we develop exponential family embeddings, a class of methods t…
A Variational Analysis of Stochastic Gradient Algorithms
Stephan Mandt, Matthew D. Hoffman, David M. Blei
Stochastic Gradient Descent (SGD) is an important algorithm in machine learning. With constant learning rates, it is a stochastic process that, after an initial phase of convergenc…
Variational Tempering
Stephan Mandt, James McInerney, Farhan Abrol +2
Variational inference (VI) combined with data subsampling enables approximate posterior inference over large data sets, but suffers from poor local optima. We first formulate a det…
Smoothed Gradients for Stochastic Variational Inference
Stephan Mandt, David Blei
Stochastic variational inference (SVI) lets us scale up Bayesian computation to massive data. It uses stochastic optimization to fit a variational distribution, following easy-to-c…