996 citations · 1.3k across the 2 of their papers we have counts for
2 papers
cs.CL2022★ 299 cited
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Shaden Smith, Mostofa Patwary, Brandon Norick +17
Pretrained general-purpose language models can achieve state-of-the-art accuracies in various natural language processing domains by adapting to downstream tasks via zero-shot, few…
cs.AI2017★ 996 cited
Mixed Precision Training
Paulius Micikevicius, Sharan Narang, Jonah Alben +8
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, th…