Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUs

Jaehwan Lee, Sangmin Lee, Chaewon Kim +2

Expert parallelism (EP) enables inference of large Mixture-of-Experts (MoE) models by placing their experts across multiple GPUs, but requires substantial communication between GPU…