64 citations · 67 across the 12 of their papers we have counts for
7 papers · 1 filter
Loss-Transformation Invariance in the Damped Newton Method
Alexander Shestakov, Sushil Bohara, Samuel Horváth +2
The Newton method is a powerful optimization algorithm, valued for its rapid local convergence and elegant geometric properties. However, its theoretical guarantees are usually lim…
Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
Artem Agafonov, Vladislav Ryspayev, Samuel Horváth +3
Quasi-Newton methods are widely used for solving convex optimization problems due to their ease of implementation, practical efficiency, and strong local convergence guarantees. Ho…
Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge
Farshed Abdukhakimov, Cuong Anh Pham, Samuel Horváth +2
The Polyak stepsize for Gradient Descent is known for its fast convergence but requires prior knowledge of the optimal functional value, which is often unavailable in practice. In…
Newton Method Revisited: Global Convergence Rates up to for Stepsize Schedules and Linesearch Procedures
Slavomír Hanzely, Farshed Abdukhakimov, Martin Takáč
This paper investigates the global convergence of stepsized Newton methods for convex functions with Hölder continuous Hessians or third derivatives. We propose several simple step…
Sketch-and-Project Meets Newton Method: Global Convergence with Low-Rank Updates
Slavomír Hanzely
In this paper, we propose the first sketch-and-project Newton method with fast global convergence rate for self-concordant functions. Our method, SGN, can be v…
Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes
Konstantin Mishchenko, Slavomír Hanzely, Peter Richtárik
In this work, we consider the problem of minimizing the sum of Moreau envelopes of given functions, which has previously appeared in the context of meta-learning and personalized f…
A Damped Newton Method Achieves Global and Local Quadratic Convergence Rate
Slavomír Hanzely, Dmitry Kamzolov, Dmitry Pasechnyuk +3
In this paper, we present the first stepsize schedule for Newton method resulting in fast global and local convergence guarantees. In particular, a) we prove an $O\left( \frac 1 {k…