351 citations · 835 across the 27 of their papers we have counts for
4 papers · 1 filter
Adaptive Gradient Descent without Descent
Yura Malitsky, Konstantin Mishchenko
We present a strikingly simple proof that two rules are sufficient to automate gradient descent: 1) don't increase the stepsize too fast and 2) don't overstep the local curvature.…
Revisiting Stochastic Extragradient
Konstantin Mishchenko, Dmitry Kovalev, Egor Shulgin +2
We fix a fundamental issue in the stochastic extragradient method by providing a new sampling strategy that is motivated by approximating implicit updates. Since the existing stoch…
Shadow Douglas--Rachford Splitting for Monotone Inclusions
Ernö Robert Csetnek, Yura Malitsky, Matthew K. Tam
In this work, we propose a new algorithm for finding a zero in the sum of two monotone operators where one is assumed to be single-valued and Lipschitz continuous. This algorithm n…
Model Function Based Conditional Gradient Method with Armijo-like Line Search
Yura Malitsky, Peter Ochs
The Conditional Gradient Method is generalized to a class of non-smooth non-convex optimization problems with many applications in machine learning. The proposed algorithm iterates…