DualPace: Decoupled Scheduling and Cooperative GPU--CPU Decoding for LLM Serving

YuXuan Lin, KeQiang Wu, Kingsum Chow

Large language model (LLM) serving relies on batching to improve inference throughput, but the key-value (KV) cache grows with sequence length and limits GPU-resident concurrency.…