Vesta: Queueing-Aware LLM Scheduling for Real-World Workloads

Lancheng Zou, Bowen Li, Wei Mao +6

Agentic applications generate multi-turn sequences of LLM requests, with each new turn extending the context accumulated from prior interactions and tool calls. Prefix caching reus…