HetCCL: Accelerating LLM Training with Heterogeneous GPUs

Heehoon Kim, Jaehwan Lee, Taejeoung Kim +6

The rapid growth of large language models is driving organizations to expand their GPU clusters, often with GPUs from multiple vendors. However, current deep learning frameworks la…