KV-RM: Regularizing KV-Cache Movement for Static-Graph LLM Serving

Zhiqing Zhong, Zhijing Ye, Jian Zhang +3

Static-graph LLM decoders provide predictable launches, fixed tensor shapes, and low submission overhead, but online decoding exposes highly irregular KV-cache behavior: request le…