13 citations · 57 across the 29 of their papers we have counts for
7 papers · 1 filter
MAST: Model-Agnostic Sparsified Training
Yury Demidovich, Grigory Malinovsky, Egor Shulgin +1
We introduce a novel optimization problem formulation that departs from the conventional way of minimizing machine learning model loss as a black-box function. Unlike traditional f…
Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
Grigory Malinovsky, Peter Richtárik, Samuel Horváth +1
Distributed learning has emerged as a leading paradigm for training large machine learning models. However, in real-world scenarios, participants may be unreliable or malicious, po…
Improving Accelerated Federated Learning with Compression and Importance Sampling
Michał Grudzień, Grigory Malinovsky, Peter Richtárik
Federated Learning is a collaborative training framework that leverages heterogeneous data distributed across a vast number of clients. Since it is practically infeasible to reques…
A Guide Through the Zoo of Biased SGD
Yury Demidovich, Grigory Malinovsky, Igor Sokolov +1
Stochastic Gradient Descent (SGD) is arguably the most important single algorithm in modern machine learning. Although SGD with unbiased gradient estimators has been studied extens…
Federated Learning with Regularized Client Participation
Grigory Malinovsky, Samuel Horváth, Konstantin Burlachenko +1
Federated Learning (FL) is a distributed machine learning approach where multiple clients work together to solve a machine learning task. One of the key challenges in FL is the iss…
TAMUNA: Doubly Accelerated Distributed Optimization under Partial Participation
Laurent Condat, Ivan Agarský, Grigory Malinovsky +1
In distributed optimization and federated learning, slow and costly communication between parallel devices and the central server constitutes the primary bottleneck. To alleviate t…