3 citations · 3 across the 3 of their papers we have counts for
27 papers
SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
Liyang Yuan, Yibo Yang, Dandan Guo +2
Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global…
TAMUNA: Doubly Accelerated Distributed Optimization under Partial Participation
Laurent Condat, Ivan Agarský, Grigory Malinovsky +1
In distributed optimization and federated learning, slow and costly communication between parallel devices and the central server constitutes the primary bottleneck. To alleviate t…
Demystifying Pipeline Parallelism: First Theory for PipeDream
Ivan Ilin, Peter Richtárik
Training modern machine learning models increasingly requires computation to be distributed across many accelerators. Data parallelism remains the default choice and is often paire…
Tighter Performance Theory of FedExProx
Wojciech Anyszka, Kaja Gruntkowska, Alexander Tyurin +1
We revisit FedExProx - a recently proposed distributed optimization method designed to enhance convergence properties of parallel proximal algorithms via extrapolation. In the proc…
CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimization
Laurent Condat, Ivan Agarský, Peter Richtárik
In distributed optimization, a large number of machines alternate between local computations and communication with a coordinating server. Communication, which can be slow and cost…
MAST: Model-Agnostic Sparsified Training
Yury Demidovich, Grigory Malinovsky, Egor Shulgin +1
We introduce a novel optimization problem formulation that departs from the conventional way of minimizing machine learning model loss as a black-box function. Unlike traditional f…