Parle: parallelizing stochastic gradient descent

Pratik Chaudhari, Carlo Baldassi, Riccardo Zecchina +3

We propose a new algorithm called Parle for parallel training of deep networks that converges 2-4x faster than a data-parallel implementation of SGD, while achieving significantly…