Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
HorizonServe: Coordinating Request Scheduling with GPU Sharing for Omni-Model Serving
Yuning Zhang, Dong Yuan
Omni models unify text, speech, image, and multimodal reasoning in a single serving backend, but this unified deployment exposes a new scheduling problem. Requests with different o…
cs.DC2026
AgentServe: Algorithm-System Co-Design for Efficient Agentic AI Serving on a Consumer-Grade GPU
Yuning Zhang, Yan Yan, Nan Yang +1
Large language models (LLMs) are increasingly deployed as AI agents that operate in short reasoning-action loops, interleaving model computation with external calls. Unlike traditi…
cs.DC2025
DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
Yuning Zhang, Grant Pinkert, Nan Yang +2
Large Language Models (LLMs) are increasingly deployed as Internet/Web services (LLM-as-a-Service) with strict latency Service-Level Objectives (SLOs) under tight GPU memory budget…