Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
ShardMeter: Sharded and Geo-Distributed Training Without the Guesswork
Tim Beringer, Patrick Diem, Felix Wolf +1
Training large-scale AI models often outgrows a single data center, demanding sharded, multi-cluster, and decentralized training. However, the huge space of resource allocations ma…
cs.DC2026
HeLoCo: Efficient asynchronous low-communication training under data and device heterogeneity
Abdullah Al Asif, Patrick Diem, Juan Pablo Muñoz +3
Distributed Low-Communication (DiLoCo) training reduces communication overhead by allowing workers to perform multiple local optimization steps before sending pseudo-gradients to a…