Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding
Jiefei Chen, Binbin Lin, Jinming Ma +9
Modern serving systems for Mixture-of-Experts (MoE) models adopt hybrid data-expert parallelism: expert parallelism (EP) shards experts across GPUs to scale capacity, while data pa…
cs.DC2025
H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
Ding Tang, Jiecheng Zhou, Jiakai Hu +5
Recent advancements in large language models (LLMs) necessitate extensive computational resources, prompting the use of diverse hardware accelerators from multiple vendors. However…