5 papers · 1 filter
Loss-Transformation Invariance in the Damped Newton Method
Alexander Shestakov, Sushil Bohara, Samuel Horváth +2
The Newton method is a powerful optimization algorithm, valued for its rapid local convergence and elegant geometric properties. However, its theoretical guarantees are usually lim…
Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
Artem Agafonov, Vladislav Ryspayev, Samuel Horváth +3
Quasi-Newton methods are widely used for solving convex optimization problems due to their ease of implementation, practical efficiency, and strong local convergence guarantees. Ho…
Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge
Farshed Abdukhakimov, Cuong Anh Pham, Samuel Horváth +2
The Polyak stepsize for Gradient Descent is known for its fast convergence but requires prior knowledge of the optimal functional value, which is often unavailable in practice. In…
Newton Method Revisited: Global Convergence Rates up to for Stepsize Schedules and Linesearch Procedures
SlavomÃr Hanzely, Farshed Abdukhakimov, Martin TakáÄ
This paper investigates the global convergence of stepsized Newton methods for convex functions with Hölder continuous Hessians or third derivatives. We propose several simple ste…
Sketch-and-Project Meets Newton Method: Global Convergence with Low-Rank Updates
SlavomÃr Hanzely
In this paper, we propose the first sketch-and-project Newton method with fast global convergence rate for self-concordant functions. Our method, SGN, can be v…