3 papers
cs.OS2026
Valve: Production Online-Offline Inference Colocation with Jointly-Bounded Preemption Latency and Rate
Fangyue Liu, Hua Liu, Xinyuan Lyu +8
LLM inference powers latency-critical production services nowadays. The bursty nature of inference traffic results in over-provisioning, which in turn leads to resource underutiliz…
cs.LG2026
Heddle: A Distributed Orchestration System for Agentic RL Rollout
Zili Zhang, Yinmin Zhong, Chengxu Yang +5
Agentic Reinforcement Learning (RL) enables LLMs to solve complex tasks by alternating between a data-collection rollout phase and a policy training phase. During rollout, the agen…
cs.DC2025
HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds
Chiheng Lou, Sheng Qi, Chao Jin +5
With the proliferation of large language model (LLM) variants, developers are turning to serverless computing for cost-efficient LLM deployment. However, public cloud providers oft…