17 citations · 61 across the 16 of their papers we have counts for
Showing 2025 · cs.CLShow all
2 papers · 2 filters
cs.CL2025
Eager Updates For Overlapped Communication and Computation in DiLoCo
Satyen Kale, Arthur Douillard, Yanislav Donchev
Distributed optimization methods such as DiLoCo have been shown to be effective in training very large models across multiple distributed workers, such as datacenters. These method…
cs.CL2025
Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch
Arthur Douillard, Yanislav Donchev, Keith Rush +11
Training of large language models (LLMs) is typically distributed across a large number of accelerators to reduce training time. Since internal states and parameter gradients need…