1 citations · 1 across the 8 of their papers we have counts for
5 papers · 1 filter
Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO
Nikolay Blagoev, Oğuzhan Ersoy, Lydia Yiyu Chen
Group Relative Policy Optimization (GRPO) has demonstrated wide adoption in the post-training of Large Language Models (LLMs). In GRPO, prompts are answered by the model and prefer…
All is Not Lost: LLM Recovery without Checkpoints
Nikolay Blagoev, Oğuzhan Ersoy, Lydia Yiyu Chen
Training LLMs on decentralized nodes or on-spot instances, lowers the training cost and enables model democratization. The inevitable challenge here is the transient churns of node…
NoLoCo: No-all-reduce Low Communication Training Method for Large Models
Jari Kolehmainen, Nikolay Blagoev, Semih Kara +3
Training large language models is generally done on clusters containing thousands of accelerators, communicating over a high-bandwidth interconnect. Scaling up these clusters is ex…
Verde: Verification via Refereed Delegation for Machine Learning Programs
Arasu Arun, Adam St. Arnaud, Alexey Titov +6
Machine learning programs, such as those performing inference, fine-tuning, and training of LLMs, are commonly delegated to untrusted compute providers. To provide correctness guar…
HDEE: Heterogeneous Domain Expert Ensemble
Oğuzhan Ersoy, Jari Kolehmainen, Gabriel Passamani Andrade
Training dense LLMs requires enormous amounts of data and centralized compute, which introduces fundamental bottlenecks and ever-growing costs for large models. Several studies aim…