104 citations · 374 across the 40 of their papers we have counts for
3 papers · 2 filters
Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks
Pratik Chaudhari, Stefano Soatto
Stochastic gradient descent (SGD) is widely believed to perform implicit regularization when used to train deep neural networks, but the precise manner in which this occurs has thu…
Parle: parallelizing stochastic gradient descent
Pratik Chaudhari, Carlo Baldassi, Riccardo Zecchina +3
We propose a new algorithm called Parle for parallel training of deep networks that converges 2-4x faster than a data-parallel implementation of SGD, while achieving significantly…
Deep Relaxation: partial differential equations for optimizing deep neural networks
Pratik Chaudhari, Adam Oberman, Stanley Osher +2
In this paper we establish a connection between non-convex optimization methods for training deep neural networks and nonlinear partial differential equations (PDEs). Relaxation te…