9 papers · 1 filter
Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization
TaeHo Yoon, Nicolas Loizou
Acceleration for deterministic root-finding problems has been extensively studied in recent years; specifically, the anchor-based, or Halpern-type methods achieve optimal convergen…
On Same-Sample and Independent-Sample Stochastic Extragradient for Monotone Variational Inequalities
TaeHo Yoon, Nicolas Loizou
We study stochastic extragradient (SEG) methods for solving monotone variational inequality problems (VIPs) over a feasible set. Although extragradient is a foundational algorithm…
Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler
Dimitris Oikonomou, Nicolas Loizou
Sharpness-Aware Minimization (SAM) has established itself as a powerful and widely adopted optimizer for training machine learning models. By explicitly minimizing the sharpness of…
Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients
Dimitris Oikonomou, Nicolas Loizou
The stochastic Polyak step size (SPS) has proven to be a promising choice for stochastic gradient descent (SGD), delivering competitive performance relative to state-of-the-art met…
Extragradient Method for -Lipschitz Root-finding Problems
Sayantan Choudhury, Nicolas Loizou
Introduced by Korpelevich in 1976, the extragradient method (EG) has become a cornerstone technique for solving min-max optimization, root-finding problems, and variational inequal…
Sharpness-Aware Minimization: General Analysis and Improved Rates
Dimitris Oikonomou, Nicolas Loizou
Sharpness-Aware Minimization (SAM) has emerged as a powerful method for improving generalization in machine learning models by minimizing the sharpness of the loss landscape. Howev…