6 citations · 14 across the 13 of their papers we have counts for
4 papers · 2 filters
Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
Ao Xiao, Bangzheng He, Baoquan Zhang +125
Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentr…
Efficient Serving of LLM Applications with Probabilistic Demand Modeling
Yifei Liu, Zuo Gan, Zhenghao Gan +8
Applications based on Large Language Models (LLMs) contains a series of tasks to address real-world problems with boosted capability, which have dynamic demand volumes on diverse b…
Themis: Efficient Sparse Model Training Through Fully Sharded Sparse Data Parallelism
Yuhao Qing, Guichao Zhu, Lintian Lei +9
Mixture-of-Experts (MoE) scales large language models cost-effectively, but expert-parallel training suffers severe straggler effects from skewed expert loads. Current systems freq…
DeepServe: Serverless Large Language Model Serving at Scale
Junhao Hu, Jiang Xu, Zhixia Liu +18
In this paper, we propose DEEPSERVE, a scalable and serverless AI platform designed to efficiently serve large language models (LLMs) at scale in cloud environments. DEEPSERVE addr…