OPEN-1B: A Fully Auditable Training Run

John Donaghy, Brian Wilcox, Oğuzhan Ersoy +6

Open-source language models have a reproducibility problem. Despite releasing weights, training data, and recipes, none of them are provably reproducible due to the non-associativi…