571 citations · 1.7k across the 33 of their papers we have counts for
3 papers · 1 filter
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal +2
The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks. These methods operate in a small-batch regime wherein a fractio…
Distributed Hessian-Free Optimization for Deep Neural Network
Xi He, Dheevatsa Mudigere, Mikhail Smelyanskiy +1
Training deep neural network is a high dimensional and a highly non-convex optimization problem. Stochastic gradient descent (SGD) algorithm and it's variations are the current sta…
Distributed Deep Learning Using Synchronous Stochastic Gradient Descent
Dipankar Das, Sasikanth Avancha, Dheevatsa Mudigere +5
We design and implement a distributed multinode synchronous SGD algorithm, without altering hyper parameters, or compressing data, or altering algorithmic behavior. We perform a de…