110 citations · 230 across the 12 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.DC2020★ 64 cited
Synthesizing Optimal Collective Algorithms
Zixian Cai, Zhengyang Liu, Saeed Maleki +4
Collective communication algorithms are an important component of distributed computation. Indeed, in the case of deep-learning, collective communication is the Amdahl's bottleneck…
cs.DC2020★ 1 cited
Scaling Distributed Training with Adaptive Summation
Saeed Maleki, Madan Musuvathi, Todd Mytkowicz +5
Stochastic gradient descent (SGD) is an inherently sequential training algorithm--computing the gradient at batch depends on the model parameters learned from batch . Prio…