Representation range needs for 16-bit neural network training

Valentina Popescu, Abhinav Venigalla, Di Wu +1

Deep learning has grown rapidly thanks to its state-of-the-art performance across a wide range of real-world applications. While neural networks have been trained using IEEE-754 bi…