5 papers · 2 filters
Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning
Kai Yi, Timur Kharisov, Igor Sokolov +1
Virtually all federated learning (FL) methods, including FedAvg, operate in the following manner: i) an orchestrating server sends the current model parameters to a cohort of clien…
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
Vladimir Malinovskii, Denis Mazur, Ivan Ilin +5
There has been significant interest in "extreme" compression of large language models (LLMs), i.e., to 1-2 bits per parameter, which allows such models to be executed efficiently o…
Sparse-ProxSkip: Accelerated Sparse-to-Sparse Training in Federated Learning
Georg Meinhardt, Kai Yi, Laurent Condat +1
In Federated Learning (FL), both client resource constraints and communication costs pose major problems for training large models. In the centralized setting, sparse training addr…
FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
Kai Yi, Nidham Gazagnadou, Peter Richtárik +1
The interest in federated learning has surged in recent research due to its unique ability to train a global model using privacy-secured information held locally on each client. Th…
FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
Kai Yi, Georg Meinhardt, Laurent Condat +1
Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with…