SLOs-Serve: Optimized Serving of Multi-SLO LLMs

Siyuan Chen, Zhipeng Jia, Samira Khan +2

This paper introduces SLOs-Serve, a system designed for serving multi-stage large language model (LLM) requests with application- and stage-specific service level objectives (SLOs)…