Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters

Runsheng Benson Guo, Utkarsh Anand, Khuzaima Daudjee +1

Large language models (LLMs) require vast amounts of GPU compute to train, but limited availability and high costs of GPUs make homogeneous clusters impractical for many organizati…