The Limit of the Batch Size

Yang You, Yuhui Wang, Huan Zhang +3

Large-batch training is an efficient approach for current distributed deep learning systems. It has enabled researchers to reduce the ImageNet/ResNet-50 training from 29 hours to a…