#kv cache
6 papers match
Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
Alexander Boesgaard Lorup
The paper investigates how the key/value cache in transformer language models affects token generation, showing that cache contents alone can reproduce divergent outputs and that n…
DualDecoder: Accelerate Long Context LLM Inference by Predictive Prefetch
Zuning Liang, Zhiyi Yao, Qi Chen +6
DualDecoder is a serving system that predicts and prefetches the key‑value cache entries needed for the next token in long‑context LLM inference, reducing GPU memory overhead and b…
A Photonic-CXL Memory Appliance for Scalable KV Cache Management in LLM Inference
Jing Ding, Yash Nishant, Chandrish Ambati +2
The paper proposes a photonic‑CXL memory appliance that uses a passive fiber shuffle to provide a 32 TB shared memory pool for large‑language‑model KV cache management, reducing la…
InferScale: GPU-Native KV Injection for Personalized LLM Serving
Peter Li, Prashant Pandey
The paper introduces InferScale, a GPU-native system that precomputes and caches key‑value (KV) representations of personalized memory facts for large language models, allowing dir…
PReM: Learning What to Preserve and When to Refresh for Context Compression
Bohan Yu, Lei Shen, Chenxi Zhou +5
The paper proposes PReM, a framework that lets language models dynamically decide which parts of a long context to keep and when to refresh stored information, improving efficiency…
[AAFLOW+] Stateful Operator Abstraction with Zero-Copy Distributed KV Cache Orchestration for Multi-Agent Workflows
Arup Kumar Sarker, Alexander James Halpern, Mills Staylor +5
The paper presents AAFLOW+, a framework that treats key‑value (KV) caches as distributed objects, enabling zero‑copy sharing of model state across multi‑agent LLM workflows to cut…