439 citations · 1.2k across the 11 of their papers we have counts for
5 papers · 1 filter
On the Generalization Benefit of Noise in Stochastic Gradient Descent
Samuel L. Smith, Erich Elsen, Soham De
It has long been argued that minibatch stochastic gradient descent can generalize better than large batch gradient descent in deep neural networks. However recent papers have quest…
AlgebraNets
Jordan Hoffmann, Simon Schmitt, Simon Osindero +2
Neural networks have historically been built layerwise from the set of functions in , i.e. with activations and weights/parameters represented…
A Practical Sparse Approximation for Real Time Recurrent Learning
Jacob Menick, Erich Elsen, Utku Evci +3
Current methods for training recurrent neural networks are based on backpropagation through time, which requires storing a complete history of network states, and prohibits updatin…
Sparse GPU Kernels for Deep Learning
Trevor Gale, Matei Zaharia, Cliff Young +1
Scientific workloads have traditionally exploited high levels of sparsity to accelerate computation and reduce memory requirements. While deep neural networks can be made sparse, a…
End-to-End Adversarial Text-to-Speech
Jeff Donahue, Sander Dieleman, Mikołaj Bińkowski +2
Modern text-to-speech synthesis pipelines typically involve multiple processing stages, each of which is designed or learnt independently from the rest. In this work, we take on th…