571 citations · 1.7k across the 33 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
Hao-Jun Michael Shi, Tsung-Hsien Lee, Shintaro Iwasaki +5
Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad family of methods for training neural networks. It constructs a block-diagonal preconditioner wh…
cs.IR2023
MTrainS: Improving DLRM training efficiency using heterogeneous memories
Hiwot Tadese Kassa, Paul Johnson, Jason Akers +7
Recommendation models are very large, requiring terabytes (TB) of memory during training. In pursuit of better quality, the model size and complexity grow over time, which requires…