Covenant-72B: Pre-Training a 72B LLM with Trustless Peers Over-the-Internet

Joel Lidin, Amir Sarfi, Erfan Miahi +6

Recently, there has been increased interest in globally distributed training, which has the promise to both reduce training costs and democratize participation in building large-sc…