TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving

Hongqiu Ni, Han Tian, Chi Zhang +2

Prefix caching introduces a fundamental tradeoff in multi-agent large language model (LLM) serving: retaining a long system-prompt key-value (KV) cache for an agent accelerates fut…