399 citations · 1.5k across the 18 of their papers we have counts for
Showing 2017Show all
2 papers · 1 filter
math.OC2017★ 66 cited
Asynchronous Decentralized Parallel Stochastic Gradient Descent
Xiangru Lian, Wei Zhang, Ce Zhang +1
Most commonly used distributed machine learning systems are either synchronous or centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a heterogene…
math.OC2017★ 399 cited
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
Xiangru Lian, Ce Zhang, Huan Zhang +3
Most distributed machine learning systems nowadays, including TensorFlow and CNTK, are built in a centralized fashion. One bottleneck of centralized algorithms lies on high communi…