2 citations · 2 across the 13 of their papers we have counts for
Showing cs.OSShow all
2 papers · 1 filter
cs.OS2026
Herschel: Continuous Optimization of Production LLM Inference through On-Demand Profiling
Luping Wang, Weigao Chen, Yifei Wu +11
Model-as-a-service platforms call for continuous optimization as complex serving conditions expose inefficiencies missed before deployment. Detailed always-on profiling can incur s…
cs.OS2026
Capture the lifecycle: KV Cache management in ReAct Agents with KVTether
Kaihua Fu, Yukun Zhou, Chaokun Chang +6
Efficient serving of long-context reasoning-and-acting (ReAct) agents relies on KV cache reuse to reduce large language model (LLM) prefill latency and monetary cost. However, a se…