18 citations · 267 across the 76 of their papers we have counts for
14 papers · 1 filter
Method with Batching for Stochastic Finite-Sum Variational Inequalities in Non-Euclidean Setting
Alexander Pichugin, Maksim Pechin, Aleksandr Beznosikov +2
Variational inequalities are a universal optimization paradigm that incorporate classical minimization and saddle point problems. Nowadays more and more tasks require to consider s…
Local SGD for Near-Quadratic Problems: Improving Convergence under Unconstrained Noise Conditions
Andrey Sadchikov, Savelii Chezhegov, Aleksandr Beznosikov +1
Distributed optimization plays an important role in modern large-scale machine learning and data processing systems by optimizing the utilization of computational resources. One of…
Average-case optimization analysis for distributed consensus algorithms on regular graphs
Nhat Trung Nguyen, Alexander Rogozin, Alexander Gasnikov
The consensus problem in distributed computing involves a network of agents aiming to compute the average of their initial vectors through local communication, represented by an un…
Decentralized Optimization with Coupled Constraints
Demyan Yarmoshik, Alexander Rogozin, Nikita Kiselev +3
We consider the decentralized minimization of a separable objective , where the variables are coupled through an affine constraint $\sum_{i=1}^n\left(\math…
Accelerated Stochastic Gradient Method with Applications to Consensus Problem in Markov-Varying Networks
Vladimir Solodkin, Savelii Chezhegov, Ruslan Nazikov +2
Stochastic optimization is a vital field in the realm of mathematical optimization, finding applications in diverse areas ranging from operations research to machine learning. In t…
Stochastic Frank-Wolfe: Unified Analysis and Zoo of Special Cases
Ruslan Nazykov, Aleksandr Shestakov, Vladimir Solodkin +3
The Conditional Gradient (or Frank-Wolfe) method is one of the most well-known methods for solving constrained optimization problems appearing in various machine learning tasks. Th…
Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
Savelii Chezhegov, Yaroslav Klyukin, Andrei Semenov +5
Methods with adaptive stepsizes, such as AdaGrad and Adam, are essential for training modern Deep Learning models, especially Large Language Models. Typically, the noise in the sto…
Local Methods with Adaptivity via Scaling
Savelii Chezhegov, Sergey Skorik, Nikolas Khachaturov +5
The rapid development of machine learning and deep learning has introduced increasingly complex optimization challenges that must be addressed. Indeed, training modern, advanced mo…
Lower Bounds and Optimal Algorithms for Non-Smooth Convex Decentralized Optimization over Time-Varying Networks
Dmitry Kovalev, Ekaterina Borodich, Alexander Gasnikov +1
We consider the task of minimizing the sum of convex functions stored in a decentralized manner across the nodes of a communication network. This problem is relatively well-studied…
Accelerated Methods with Compression for Horizontal and Vertical Federated Learning
Sergey Stanko, Timur Karimullin, Aleksandr Beznosikov +1
Distributed optimization algorithms have emerged as a superior approaches for solving machine learning problems. To accommodate the diverse ways in which data can be stored across…
Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning
Andrei Semenov, Vladimir Ivanov, Aleksandr Beznosikov +1
We propose a novel architecture and method of explainable classification with Concept Bottleneck Models (CBMs). While SOTA approaches to Image Classification task work as a black b…
Extragradient Sliding for Composite Non-Monotone Variational Inequalities
Roman Emelyanov, Andrey Tikhomirov, Aleksandr Beznosikov +1
Variational inequalities offer a versatile and straightforward approach to analyzing a broad range of equilibrium problems in both theoretical and practical fields. In this paper,…
Decentralized Finite-Sum Optimization over Time-Varying Networks
Dmitry Metelev, Savelii Chezhegov, Alexander Rogozin +4
We consider decentralized time-varying stochastic optimization problems where each of the functions held by the nodes has a finite sum structure. Such problems can be efficiently s…
Optimal Data Splitting in Distributed Optimization for Machine Learning
Daniil Medyakov, Gleb Molodtsov, Aleksandr Beznosikov +1
The distributed optimization problem has become increasingly relevant recently. It has a lot of advantages such as processing a large amount of data in less time compared to non-di…