2 papers
cs.DC2025
Serving Large Language Models on Huawei CloudMatrix384
Pengfei Zuo, Huimin Lin, Junbo Deng +43
The rapid evolution of large language models (LLMs), driven by growing parameter scales, adoption of mixture-of-experts (MoE) architectures, and expanding context lengths, imposes…
cs.DC2024
P/D-Serve: Serving Disaggregated Large Language Model at Scale
Yibo Jin, Tao Wang, Huimin Lin +27
Serving disaggregated large language models (LLMs) over tens of thousands of xPU devices (GPUs or NPUs) with reliable performance faces multiple challenges. 1) Ignoring the diversi…