Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving

Shrikara Arun, Anjaly Parayil, Srikant Bharadwaj +2

Disaggregated LLM serving runs prefill and decode on separate GPU pools to keep the two phases from interfering. In practice, this creates a new asymmetry: under bursty, heavy-tail…