2 papers
cs.DC2026
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
cs.LG2025
BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures
Xiannan Hu, Tianyou Zeng, Xiaoming Yuan +3
Serving large language models (LLMs) to millions of users requires efficient resource allocation and parallelism strategies. It is a labor intensive trial-and-error process to find…