1 citations · 1 across the 5 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024★ 1 cited
Photon: Federated LLM Pre-Training
Lorenzo Sani, Alex Iacob, Zeyu Cao +8
Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distr…
cs.LG2024
DEPT: Decoupled Embeddings for Pre-training Language Models
Alex Iacob, Lorenzo Sani, Meghdad Kurmanji +5
Language Model pre-training uses broad data mixtures to enhance performance across domains and languages. However, training on such heterogeneous text corpora requires extensive an…