Showing cs.DCShow all
2 papers · 1 filter
cs.DC2025
CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
Jiashun Suo, Xiaojian Liao, Limin Xiao +4
Large language models like GPT-4 are resource-intensive, but recent advancements suggest that smaller, specialized experts can outperform the monolithic models on specific tasks. T…
cs.DC2025
PipeBoost: Resilient Pipelined Architecture for Fast Serverless LLM Scaling
Chongpeng Liu, Xiaojian Liao, Hancheng Liu +2
This paper presents PipeBoost, a low-latency LLM serving system for multi-GPU (serverless) clusters, which can rapidly launch inference services in response to bursty requests with…