18 citations · 267 across the 76 of their papers we have counts for
6 papers · 1 filter
Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
Savelii Chezhegov, Yaroslav Klyukin, Andrei Semenov +5
Methods with adaptive stepsizes, such as AdaGrad and Adam, are essential for training modern Deep Learning models, especially Large Language Models. Typically, the noise in the sto…
Local Methods with Adaptivity via Scaling
Savelii Chezhegov, Sergey Skorik, Nikolas Khachaturov +5
The rapid development of machine learning and deep learning has introduced increasingly complex optimization challenges that must be addressed. Indeed, training modern, advanced mo…
On Scaled Methods for Saddle Point Problems
Aleksandr Beznosikov, Aibek Alanov, Dmitry Kovalev +2
Methods with adaptive scaling of different features play a key role in solving saddle point problems, primarily due to Adam's popularity for solving adversarial machine learning pr…
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees
Aleksandr Beznosikov, Peter Richtárik, Michael Diskin +2
Variational inequalities in general and saddle point problems in particular are increasingly relevant in machine learning applications, including adversarial learning, GANs, transp…
Decentralized Personalized Federated Learning for Min-Max Problems
Ekaterina Borodich, Aleksandr Beznosikov, Abdurakhmon Sadiev +4
Personalized Federated Learning (PFL) has witnessed remarkable advancements, enabling the development of innovative machine learning applications that preserve the privacy of train…
Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms
Aleksandr Beznosikov, Valentin Samokhin, Alexander Gasnikov
This paper focuses on the distributed optimization of stochastic saddle point problems. The first part of the paper is devoted to lower bounds for the centralized and decentralized…