1 citations · 1 across the 1 of their papers we have counts for
3 papers
KV Cache Transform Coding for Compact Storage in LLM Inference
Konrad Staniszewski, Adrian ÅaÅcucki
Serving large language models (LLMs) at scale necessitates efficient key-value (KV) cache management. KV caches can be reused across conversation turns via shared-prefix prompts th…
Inference-Time Hyper-Scaling with KV Cache Compression
Adrian ÅaÅcucki, Konrad Staniszewski, Piotr Nawrot +1
Inference-time scaling trades efficiency for increased reasoning accuracy by generating longer or more parallel sequences. However, in Transformer LLMs, generation cost is bottlene…
Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference
Piotr Nawrot, Adrian ÅaÅcucki, Marcin Chochowski +2
Transformers have emerged as the backbone of large language models (LLMs). However, generation remains inefficient due to the need to store in memory a cache of key-value represent…