3 citations · 5 across the 5 of their papers we have counts for
5 papers
AdamL: A fast adaptive gradient method incorporating loss function
Lu Xia, Stefano Massei
Adaptive first-order optimizers are fundamental tools in deep learning, although they may suffer from poor generalization due to the nonuniform gradient scaling. In this work, we p…
On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality
Lu Xia, Michiel E. Hochstenbach, Stefano Massei
When training neural networks with low-precision computation, rounding errors often cause stagnation or are detrimental to the convergence of the optimizers; in this paper we study…
On the influence of stochastic roundoff errors and their bias on the convergence of the gradient descent method with low-precision floating-point computation
Lu Xia, Stefano Massei, Michiel E. Hochstenbach +1
When implementing the gradient descent method in low precision, the employment of stochastic rounding schemes helps to prevent stagnation of convergence caused by the vanishing gra…
A Simple and Efficient Stochastic Rounding Method for Training Neural Networks in Low Precision
Lu Xia, Martijn Anthonissen, Michiel Hochstenbach +1
Conventional stochastic rounding (CSR) is widely employed in the training of neural networks (NNs), showing promising training results even in low-precision computations. We introd…
Improved stochastic rounding
Lu Xia, Martijn Anthonissen, Michiel Hochstenbach +1
Due to the limited number of bits in floating-point or fixed-point arithmetic, rounding is a necessary step in many computations. Although rounding methods can be tailored for diff…