32 citations · 75 across the 10 of their papers we have counts for
3 papers · 2 filters
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model
Guodong Zhang, Lala Li, Zachary Nado +5
Increasing the batch size is a popular way to speed up neural network training, but beyond some critical batch size, larger batch sizes yield diminishing returns. In this work, we…
Differentiable Game Mechanics
Alistair Letcher, David Balduzzi, Sebastien Racaniere +4
Deep learning is built on the foundational guarantee that gradient descent on an objective function converges to local minima. Unfortunately, this guarantee fails in settings, such…
On the Variance of Unbiased Online Recurrent Optimization
Tim Cooijmans, James Martens
The recently proposed Unbiased Online Recurrent Optimization algorithm (UORO, arXiv:1702.05043) uses an unbiased approximation of RTRL to achieve fully online gradient-based learni…