NBCLT: Parameter-Agnostic Norm-Ball Clipped Local Training

Linxuan Pan, Shucheng Yang, Junchi Yang

Distributed learning with multiple local steps reduces communication but aggravates drift, and its practical performance often depends on carefully tuned hyperparameters under hete…