NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding

Jiefei Chen, Binbin Lin, Jinming Ma +9

Modern serving systems for Mixture-of-Experts (MoE) models adopt hybrid data-expert parallelism: expert parallelism (EP) shards experts across GPUs to scale capacity, while data pa…