996 citations · 2.4k across the 62 of their papers we have counts for
5 papers · 1 filter
Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification
Igor Gitman, Boris Ginsburg
Batch normalization (BN) has become a de facto standard for training deep convolutional networks. However, BN accounts for a significant fraction of training run-time and is diffic…
Mixed Precision Training
Paulius Micikevicius, Sharan Narang, Jonah Alben +8
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, th…
Large Batch Training of Convolutional Networks
Yang You, Igor Gitman, Boris Ginsburg
A common way to speed up training of large convolutional networks is to add computational units. Training is then performed using data-parallel synchronous Stochastic Gradient Desc…
Training Deep AutoEncoders for Collaborative Filtering
Oleksii Kuchaiev, Boris Ginsburg
This paper proposes a novel model for the rating prediction task in recommender systems which significantly outperforms previous state-of-the art models on a time-split Netflix dat…
Factorization tricks for LSTM networks
Oleksii Kuchaiev, Boris Ginsburg
We present two simple ways of reducing the number of parameters and accelerating the training of large Long Short-Term Memory (LSTM) networks: the first one is "matrix factorizatio…