527 citations · 1.7k across the 34 of their papers we have counts for
Showing 2016 · stat.MLShow all
2 papers · 2 filters
stat.ML2016★ 78 cited
Parallel SGD: When does averaging help?
Jian Zhang, Christopher De Sa, Ioannis Mitliagkas +1
Consider a number of workers running SGD independently on the same pool of data and averaging the models every once in a while -- a common but not well understood practice. We stud…
stat.ML2016
Asynchrony begets Momentum, with an Application to Deep Learning
Ioannis Mitliagkas, Ce Zhang, Stefan Hadjis +1
Asynchronous methods are widely used in deep learning, but have limited theoretical justification when applied to non-convex problems. We show that running stochastic gradient desc…