works on
dual-token decoding 1kv cache 1llm serving 1long-context inference 1predictive prefetch 1sparse retrieval 1

From the 1 of 1 linked paper with an AI index.

collaborators

1 paper

cs.DC2026

DualDecoder: Accelerate Long Context LLM Inference by Predictive Prefetch

Zuning Liang, Zhiyi Yao, Qi Chen +6

DualDecoder is a serving system that predicts and prefetches the key‑value cache entries needed for the next token in long‑context LLM inference, reducing GPU memory overhead and b…