17 citations · 39 across the 35 of their papers we have counts for
11 papers · 1 filter
Photon: Federated LLM Pre-Training
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distr…
DEPT: Decoupled Embeddings for Pre-training Language Models
Alex Iacob, Lorenzo Sani, Meghdad Kurmanji +5
Language Model pre-training uses broad data mixtures to enhance performance across domains and languages. However, training on such heterogeneous text corpora requires extensive an…
Small Language Models: Survey, Measurements, and Insights
Zhenyan Lu, Xiang Li, Dongqi Cai +5
Small language models (SLMs), despite their widespread adoption in modern smart devices, have received significantly less academic attention compared to their large language model…
How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective
Xinchi Qiu, William F. Shen, Yihong Chen +4
While unlearning knowledge from large language models (LLMs) is receiving increasing attention, one important aspect remains unexplored. Existing approaches and benchmarks assume d…
Compliance Cards: Automated EU AI Act Compliance Analyses amidst a Complex AI Supply Chain
Bill Marino, Yaqub Chaudhary, Yulu Pi +6
As the AI supply chain grows more complex, AI systems and models are increasingly likely to incorporate multiple internally- or externally-sourced components such as datasets and (…
Sheaf HyperNetworks for Personalized Federated Learning
Bao Nguyen, Lorenzo Sani, Xinchi Qiu +2
Graph hypernetworks (GHNs), constructed by combining graph neural networks (GNNs) with hypernetworks (HNs), leverage relational data across various domains such as neural architect…
Worldwide Federated Training of Language Models
Alex Iacob, Lorenzo Sani, Bill Marino +3
The reliance of language model training on massive amounts of computation and vast datasets scraped from potentially low-quality, copyrighted, or sensitive data has come into quest…
The Future of Large Language Model Pre-training is Federated
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Generative pre-trained large language models (LLMs) have demonstrated impressive performance over a wide range of tasks, thanks to the unprecedented amount of data they have been t…
Attacks on Third-Party APIs of Large Language Models
Wanru Zhao, Vidit Khazanchi, Haodi Xing +3
Large language model (LLM) services have recently begun offering a plugin ecosystem to interact with third-party API services. This innovation enhances the capabilities of LLMs, bu…
Federated Learning Priorities Under the European Union Artificial Intelligence Act
Herbert Woisetschläger, Alexander Erben, Bill Marino +4
The age of AI regulation is upon us, with the European Union Artificial Intelligence Act (AI Act) leading the way. Our key inquiry is how this will affect Federated Learning (FL),…
FedGuCci: Making Local Models More Connected in Landscape for Federated Learning
Zexi Li, Jie Lin, Zhiqi Li +5
Federated learning (FL) involves multiple heterogeneous clients collaboratively training a global model via iterative local updates and model fusion. The generalization of FL's glo…