NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference

Mubarak Adetunji Ojewale

Disaggregated LLM inference forces the KV cache to traverse the datacenter network before decoding begins, so transfer time enters directly into the Time to First Token (TTFT) budg…