3 papers
cs.DC2026
DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
Ying Yuan, Pengfei Zuo, Bo Wang +3
In LLM serving, reusing the KV cache of prompts across requests is critical for reducing TTFT and serving costs. Cache-affinity scheduling, which co-locates requests with the same…
cs.DC2025
Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
Chao Wang, Pengfei Zuo, Zhangyu Chen +3
An ongoing debate considers whether prefill-decode (PD) aggregation or disaggregation is superior for serving large language models (LLMs). This has driven optimizations for both a…
cs.DC2025
Serving Large Language Models on Huawei CloudMatrix384
Pengfei Zuo, Huimin Lin, Junbo Deng +43
The rapid evolution of large language models (LLMs), driven by growing parameter scales, adoption of mixture-of-experts (MoE) architectures, and expanding context lengths, imposes…