198 citations · 201 across the 2 of their papers we have counts for
2 papers
cs.LG2019★ 3 cited
Improving the convergence of SGD through adaptive batch sizes
Scott Sievert, Shrey Shah
Mini-batch stochastic gradient descent (SGD) and variants thereof approximate the objective function's gradient with a small number of training examples, aka the batch size. Small…
stat.ML2018★ 198 cited
ATOMO: Communication-efficient Learning via Atomic Sparsification
Hongyi Wang, Scott Sievert, Zachary Charles +3
Distributed model training suffers from communication overheads due to frequent gradient updates transmitted between compute nodes. To mitigate these overheads, several studies pro…