Efficient Remote KV Cache Reuse with GPU-native Video Codec

Liang Mi, Weijun Wang, Jinghan Chen +3

Remote KV cache reuse fetches KV cache for identical contexts from remote storage, avoiding recomputation, accelerating LLM inference. While it excels in high-speed networks, its p…