Approximation Algorithms for Dual-Memory Request Scheduling in Disaggregated LLM Inference

Xin Wang, Hong Shen, Hui Tian +1

Disaggregated LLM inference executes the prefill and decode phases on separate resource pools, reducing interference between the two phases. However, this architecture introduces a…