Consistent Lock-free Parallel Stochastic Gradient Descent for Fast and Stable Convergence

Karl Bäckström, Ivan Walulya, Marina Papatriantafilou +1

Stochastic gradient descent (SGD) is an essential element in Machine Learning (ML) algorithms. Asynchronous parallel shared-memory SGD (AsyncSGD), including synchronization-free al…