1 citations · 1 across the 8 of their papers we have counts for
5 papers · 1 filter
OPEN-1B: A Fully Auditable Training Run
John Donaghy, Brian Wilcox, Oğuzhan Ersoy +6
Open-source language models have a reproducibility problem. Despite releasing weights, training data, and recipes, none of them are provably reproducible due to the non-associativi…
The Role of Feedback Alignment in Self-Distillation
Semih Kara, Oğuzhan Ersoy
Conditioning a language model on additional context, such as feedback on a previous attempt, typically improves its response. Self-distillation trains the model to retain this impr…
IR3DE: A Linear Router for Large Language Models
Eros Fanì, Oğuzhan Ersoy
Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LL…
F-TIS: Harnessing Diverse Models in Collaborative GRPO
Nikolay Blagoev, Oğuzhan Ersoy, Wendelin Boehmer +1
Reinforcement learning methods such as GRPO have seen great popularity in LLM post-training. In GRPO, models produce completions to a set of prompts, which are rewarded, and the po…
Backdoor Attacks on Decentralised Post-Training
Oğuzhan Ersoy, Nikolay Blagoev, Jona te Lintelo +3
Decentralised post-training of large language models utilises data and pipeline parallelism techniques to split the data and the model. Unfortunately, decentralised post-training c…