114 citations · 664 across the 92 of their papers we have counts for
35 papers · 1 filter
Bregman meets Lévy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time
Pierre-Louis Cauvin, Panayotis Mertikopoulos
We study the robustness of stochastic mirror descent (SMD) under heavy-tailed noise, focusing on whether the method retains its convergence guarantees when run with infinite-varian…
Setwise Coordinate Descent for Dual Asynchronous Decentralized Optimization
Marina Costantini, Nikolaos Liakopoulos, Panayotis Mertikopoulos +1
In decentralized optimization over networks, synchronizing the updates of all nodes incurs significant communication overhead. For this reason, much of the recent literature has fo…
The global convergence time of stochastic gradient descent in non-convex landscapes: Sharp estimates via large deviations
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we examine the time it takes for stochastic gradient descent (SGD) to reach the global minimum of a general, non-convex loss function. We approach this question thro…
What is the long-run distribution of stochastic gradient descent? A large deviations analysis
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we examine the long-run distribution of stochastic gradient descent (SGD) in general, non-convex problems. Specifically, we seek to understand which regions of the p…
Riemannian stochastic optimization methods avoid strict saddle points
Ya-Ping Hsieh, Mohammad Reza Karimi, Andreas Krause +1
Many modern machine learning applications - from online principal component analysis to covariance matrix identification and dictionary learning - can be formulated as minimization…
The rate of convergence of Bregman proximal methods: Local geometry vs. regularity vs. sharpness
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
We examine the last-iterate convergence rate of Bregman proximal methods - from mirror descent to mirror-prox and its optimistic variants - as a function of the local geometry indu…