Stochastic Runge-Kutta methods and adaptive SGD-G2 stochastic gradient descent

Imen Ayadi, Gabriel Turinici

The minimization of the loss function is of paramount importance in deep neural networks. On the other hand, many popular optimization algorithms have been shown to correspond to s…