most citedActivations and Gradients Compression for Model-Parallel Training

6 citations · 11 across the 7 of their papers we have counts for

collaborators

7 papers

math.OC20241 cited

Accelerated Stochastic Gradient Method with Applications to Consensus Problem in Markov-Varying Networks

Vladimir Solodkin, Savelii Chezhegov, Ruslan Nazikov +2

Stochastic optimization is a vital field in the realm of mathematical optimization, finding applications in diverse areas ranging from operations research to machine learning. In t…

math.OC2024

Accelerated Methods with Compression for Horizontal and Vertical Federated Learning

Sergey Stanko, Timur Karimullin, Aleksandr Beznosikov +1

Distributed optimization algorithms have emerged as a superior approaches for solving machine learning problems. To accommodate the diverse ways in which data can be stored across…

cs.CV2024

Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning

Andrei Semenov, Vladimir Ivanov, Aleksandr Beznosikov +1

We propose a novel architecture and method of explainable classification with Concept Bottleneck Models (CBMs). While SOTA approaches to Image Classification task work as a black b…

math.OC2024

Optimal Analysis of Method with Batching for Monotone Stochastic Finite-Sum Variational Inequalities

Alexander Pichugin, Maksim Pechin, Aleksandr Beznosikov +1

Variational inequalities are a universal optimization paradigm that is interesting in itself, but also incorporates classical minimization and saddle point problems. Modern realiti…

math.OC20243 cited

Optimal Data Splitting in Distributed Optimization for Machine Learning

Daniil Medyakov, Gleb Molodtsov, Aleksandr Beznosikov +1

The distributed optimization problem has become increasingly relevant recently. It has a lot of advantages such as processing a large amount of data in less time compared to non-di…

cs.LG20246 cited

Activations and Gradients Compression for Model-Parallel Training

Mikhail Rudakov, Aleksandr Beznosikov, Yaroslav Kholodov +1

Large neural networks require enormous computational clusters of machines. Model-parallel training, when the model architecture is partitioned sequentially between workers, is a po…