4 citations · 7 across the 9 of their papers we have counts for
6 papers · 1 filter
Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment
Yan Gao, Mohammad Naseri, Javier Fernandez-Marques +19
Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Exi…
FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs
Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji +5
Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly coupled hardware accelerators. Mixture-of-Experts (MoEs) architectures partially…
Photon: Federated LLM Pre-Training
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distr…
DEPT: Decoupled Embeddings for Pre-training Language Models
Alex Iacob, Lorenzo Sani, Meghdad Kurmanji +5
Language Model pre-training uses broad data mixtures to enhance performance across domains and languages. However, training on such heterogeneous text corpora requires extensive an…
The Future of Large Language Model Pre-training is Federated
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Generative pre-trained large language models (LLMs) have demonstrated impressive performance over a wide range of tasks, thanks to the unprecedented amount of data they have been t…
FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients
Xinchi Qiu, Yan Gao, Lorenzo Sani +6
Federated learning (FL) is a distributed learning paradigm that facilitates collaborative training of a shared global model across devices while keeping data localized. The deploym…