365 citations · 537 across the 15 of their papers we have counts for
1 paper · 2 filters
Arthur Douillard, Yanislav Donchev, Keith Rush +11
Training of large language models (LLMs) is typically distributed across a large number of accelerators to reduce training time. Since internal states and parameter gradients need…