Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines

Marcel Wagenländer, Otto White, Britannio Jarrett +6

Agentic workflows carry out complex tasks by orchestrating multiple large language models (LLMs) and tools. Serving such workflows at a target throughput with low latency is challe…