11 citations · 24 across the 6 of their papers we have counts for
4 papers · 1 filter
Nonlinear MCMC for Bayesian Machine Learning
James Vuckovic
We explore the application of a nonlinear MCMC technique first introduced in [1] to problems in Bayesian machine learning. We provide a convergence guarantee in total variation tha…
On the Regularity of Attention
James Vuckovic, Aristide Baratin, Remi Tachet des Combes
Attention is a powerful component of modern neural networks across a wide variety of domains. In this paper, we seek to quantify the regularity (i.e. the amount of smoothness) of t…
A Mathematical Theory of Attention
James Vuckovic, Aristide Baratin, Remi Tachet des Combes
Attention is a powerful component of modern neural networks across a wide variety of domains. However, despite its ubiquity in machine learning, there is a gap in our understanding…
Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization
James Vuckovic
We introduce Kalman Gradient Descent, a stochastic optimization algorithm that uses Kalman filtering to adaptively reduce gradient variance in stochastic gradient descent by filter…