ABS: Adaptive Bounded Staleness Converges Faster and Communicates Less

Qiao Tan, Feng Zhu, Jingjing Zhang

Wall-clock convergence time and communication rounds are critical performance metrics in distributed learning with parameter-server setting. While synchronous methods converge fast…