507 citations · 563 across the 3 of their papers we have counts for
3 papers
cs.LG2018
Convergence Analysis of Gradient Descent Algorithms with Proportional Updates
Igor Gitman, Deepak Dilipkumar, Ben Parr
The rise of deep learning in recent years has brought with it increasingly clever optimization methods to deal with complex, non-linear loss functions. These methods are often desi…
cs.CV2017★ 56 cited
Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification
Igor Gitman, Boris Ginsburg
Batch normalization (BN) has become a de facto standard for training deep convolutional networks. However, BN accounts for a significant fraction of training run-time and is diffic…
cs.CV2017★ 507 cited
Large Batch Training of Convolutional Networks
Yang You, Igor Gitman, Boris Ginsburg
A common way to speed up training of large convolutional networks is to add computational units. Training is then performed using data-parallel synchronous Stochastic Gradient Desc…