collaborators

1 paper

cs.DC2026

HeLoCo: Efficient asynchronous low-communication training under data and device heterogeneity

Abdullah Al Asif, Patrick Diem, Juan Pablo Muñoz +3

Distributed Low-Communication (DiLoCo) training reduces communication overhead by allowing workers to perform multiple local optimization steps before sending pseudo-gradients to a…