Overshoot: Taking advantage of future gradients in momentum-based stochastic optimization

Jakub Kopal, Michal Gregor, Santiago de Leon-Martinez +1

Overshoot is a novel, momentum-based stochastic gradient descent optimization method designed to enhance performance beyond standard and Nesterov's momentum. In conventional moment…