2 papers
cs.DC2026
Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUs
Jaehwan Lee, Sangmin Lee, Chaewon Kim +2
Expert parallelism (EP) enables inference of large Mixture-of-Experts (MoE) models by placing their experts across multiple GPUs, but requires substantial communication between GPU…
cs.DC2026
HetCCL: Accelerating LLM Training with Heterogeneous GPUs
Heehoon Kim, Jaehwan Lee, Taejeoung Kim +6
The rapid growth of large language models is driving organizations to expand their GPU clusters, often with GPUs from multiple vendors. However, current deep learning frameworks la…