Local MixVR: Breaking the Communication-Sample Dependence in Distributed Learning

Tehila Dahan, Bassel Hamoud, Roie Reshef +2

Communication overhead is a crucial bottleneck in scalable distributed learning. While existing methods aim to efficiently utilize data points, such as Local SGD, Minibatch SGD, an…