8 citations · 10 across the 2 of their papers we have counts for
3 papers
stat.ML2021★ 2 cited
On the Regularity of Attention
James Vuckovic, Aristide Baratin, Remi Tachet des Combes
Attention is a powerful component of modern neural networks across a wide variety of domains. In this paper, we seek to quantify the regularity (i.e. the amount of smoothness) of t…
stat.ML2020★ 8 cited
A Mathematical Theory of Attention
James Vuckovic, Aristide Baratin, Remi Tachet des Combes
Attention is a powerful component of modern neural networks across a wide variety of domains. However, despite its ubiquity in machine learning, there is a gap in our understanding…
stat.ML2018
Kalman Gradient Descent: Adaptive Variance Reduction in Stochastic Optimization
James Vuckovic
We introduce Kalman Gradient Descent, a stochastic optimization algorithm that uses Kalman filtering to adaptively reduce gradient variance in stochastic gradient descent by filter…