124 citations · 144 across the 14 of their papers we have counts for
9 papers · 1 filter
FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEs
Lorenzo Sani, Zeyu Cao, Meghdad Kurmanji +5
Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly coupled hardware accelerators. Mixture-of-Experts (MoEs) architectures partially…
Beyond Scaling: Agents Are Heading to the Edge
Chunlin Tian, Dongqi Cai, Wanru Zhao +1
The bottleneck of useful agentic intelligence has shifted from compressing world knowledge into a single model to executing a coordinated system. This position paper argues that pe…
Photon: Federated LLM Pre-Training
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distr…
Data Quality Control in Federated Instruction-tuning of Large Language Models
Yaxin Du, Rui Ye, Fengting Yuchi +4
Federated Learning (FL) enables privacy-preserving collaborative instruction tuning of large language models (LLMs) by leveraging massively distributed data. However, the decentral…
The Future of Large Language Model Pre-training is Federated
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Generative pre-trained large language models (LLMs) have demonstrated impressive performance over a wide range of tasks, thanks to the unprecedented amount of data they have been t…
Enhancing Data Quality in Federated Fine-Tuning of Foundation Models
Wanru Zhao, Yaxin Du, Nicholas Donald Lane +2
In the current landscape of foundation model training, there is a significant reliance on public domain data, which is nearing exhaustion according to recent research. To further s…
FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients
Xinchi Qiu, Yan Gao, Lorenzo Sani +6
Federated learning (FL) is a distributed learning paradigm that facilitates collaborative training of a shared global model across devices while keeping data localized. The deploym…
Efficient Vertical Federated Learning with Secure Aggregation
Xinchi Qiu, Heng Pan, Wanru Zhao +3
The majority of work in privacy-preserving federated learning (FL) has been focusing on horizontally partitioned datasets where clients share the same sets of features and can trai…
Protea: Client Profiling within Federated Systems using Flower
Wanru Zhao, Xinchi Qiu, Javier Fernandez-Marques +2
Federated Learning (FL) has emerged as a prospective solution that facilitates the training of a high-performing centralised model without compromising the privacy of users. While…