DBS: Dynamic Batch Size For Distributed Deep Neural Network Training

Qing Ye, Yuhao Zhou, Mingjia Shi +2

Synchronous strategies with data parallelism, such as the Synchronous StochasticGradient Descent (S-SGD) and the model averaging methods, are widely utilizedin distributed training…