8 papers · 1 filter
Broximal Gradient Descent: A Projection-Free Sister of Projected Gradient Descent
Peter Richtárik, Kaja Gruntkowska, Hanmin Li
We propose Broximal Gradient Descent (BroxGD), a projection-free sister method to projected gradient descent for constrained optimization. Its forward--backward construction replac…
Broximal Alignment for Global Non-Convex Optimization
Kaja Gruntkowska, Hanmin Li, Xun Qian +1
Most non-convex optimization theory is built around gradient dynamics, leaving global convergence largely unexplored. The dominant paradigm focuses on stationarity, certifying only…
Stabilized Proximal Point Method via Trust Region Control
Hanmin Li, Kaja Gruntkowska, Peter Richtárik
The Proximal Point Method (PPM) (Rockafellar, 1976) is a fundamental tool for nonsmooth convex optimization. However, its convergence is not linear under general convexity in the a…
The Ball-Proximal (="Broximal") Point Method: a New Algorithm, Convergence Theory, and Applications
Kaja Gruntkowska, Hanmin Li, Aadi Rane +1
Non-smooth and non-convex global optimization poses significant challenges across various applications, where standard gradient-based methods often struggle. We propose the Ball-Pr…
On the Convergence of FedProx with Extrapolation and Inexact Prox
Hanmin Li, Peter Richtárik
Enhancing the FedProx federated learning algorithm (Li et al., 2020) with server-side extrapolation, Li et al. (2024a) recently introduced the FedExProx method. Their theoretical a…
The Power of Extrapolation in Federated Learning
Hanmin Li, Kirill Acharya, Peter Richtárik
We propose and study several server-extrapolation strategies for enhancing the theoretical and empirical convergence properties of the popular federated learning optimizer FedProx…
Variance Reduced Distributed Non-Convex Optimization Using Matrix Stepsizes
Hanmin Li, Avetik Karagulyan, Peter Richtárik
Matrix-stepsized gradient descent algorithms have been shown to have superior performance in non-convex optimization problems compared to their scalar counterparts. The det-CGD alg…
Det-CGD: Compressed Gradient Descent with Matrix Stepsizes for Non-Convex Optimization
Hanmin Li, Avetik Karagulyan, Peter Richtárik
This paper introduces a new method for minimizing matrix-smooth non-convex objectives through the use of novel Compressed Gradient Descent (CGD) algorithms enhanced with a matrix-v…