Showing math.OCShow all
2 papers · 1 filter
math.OC2020
Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance
A. Barakat, P. Bianchi, W. Hachem +1
In this paper, a general stochastic optimization procedure is studied, unifying several variants of the stochastic gradient descent such as, among others, the stochastic heavy ball…
math.OC2019
Convergence Analysis of a Momentum Algorithm with Adaptive Step Size for Non Convex Optimization
Anas Barakat, Pascal Bianchi
Although ADAM is a very popular algorithm for optimizing the weights of neural networks, it has been recently shown that it can diverge even in simple convex optimization examples.…