Mixed Precision Training

Paulius Micikevicius, Sharan Narang, Jonah Alben +8

Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, th…