12 citations · 12 across the 5 of their papers we have counts for
5 papers
Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge
Farshed Abdukhakimov, Cuong Anh Pham, Samuel Horváth +2
The Polyak stepsize for Gradient Descent is known for its fast convergence but requires prior knowledge of the optimal functional value, which is often unavailable in practice. In…
Newton Method Revisited: Global Convergence Rates up to for Stepsize Schedules and Linesearch Procedures
Slavomír Hanzely, Farshed Abdukhakimov, Martin Takáč
This paper investigates the global convergence of stepsized Newton methods for convex functions with Hölder continuous Hessians or third derivatives. We propose several simple step…
SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
Farshed Abdukhakimov, Chulu Xiang, Dmitry Kamzolov +2
Adaptive optimization methods are widely recognized as among the most popular approaches for training Deep Neural Networks (DNNs). Techniques such as Adam, AdaGrad, and AdaHessian…
Stochastic Gradient Descent with Preconditioned Polyak Step-size
Farshed Abdukhakimov, Chulu Xiang, Dmitry Kamzolov +1
Stochastic Gradient Descent (SGD) is one of the many iterative optimization methods that are widely used in solving machine learning problems. These methods display valuable proper…
The Power of First-Order Smooth Optimization for Black-Box Non-Smooth Problems
Alexander Gasnikov, Anton Novitskii, Vasilii Novitskii +6
Gradient-free/zeroth-order methods for black-box convex optimization have been extensively studied in the last decade with the main focus on oracle calls complexity. In this paper,…