A Simple and Efficient Stochastic Rounding Method for Training Neural Networks in Low Precision

Lu Xia, Martijn Anthonissen, Michiel Hochstenbach +1

Conventional stochastic rounding (CSR) is widely employed in the training of neural networks (NNs), showing promising training results even in low-precision computations. We introd…