Go With The Flow: Churn-Tolerant Decentralized Training of Large Language Models

Nikolay Blagoev, Bart Cox, Jérémie Decouchant +1

Motivated by the emergence of large language models (LLMs) and the importance of democratizing their training, we propose GWTF, the first crash tolerant practical decentralized tra…