SLO-Aware Compute Resource Allocation for Prefill-Decode Disaggregated LLM Inference

Luchang Li, Dongfang Li, Bozhao Gong +1

Prefill-Decode (P/D) disaggregation has emerged as a widely adopted optimization strategy for Large Language Model (LLM) inference. However, there currently exists no well-establis…