507 citations · 623 across the 3 of their papers we have counts for
4 papers
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
Yang You, Jing Li, Sashank Reddi +7
Training large deep neural networks on massive datasets is computationally very challenging. There has been recent surge in interest in using large batch stochastic optimization me…
Large Batch Training of Convolutional Networks
Yang You, Igor Gitman, Boris Ginsburg
A common way to speed up training of large convolutional networks is to add computational units. Training is then performed using data-parallel synchronous Stochastic Gradient Desc…
ImageNet Training in Minutes
Yang You, Zhao Zhang, Cho-Jui Hsieh +2
Finishing 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIA M40 GPU takes 14 days. This training requires 10^18 single precision operations in total. On the other hand, the…
Scaling Deep Learning on GPU and Knights Landing clusters
Yang You, Aydin Buluc, James Demmel
The speed of deep neural networks training has become a big bottleneck of deep learning research and development. For example, training GoogleNet by ImageNet dataset on one Nvidia…