111 citations · 111 across the 6 of their papers we have counts for
1 paper · 1 filter
Joao Monteiro, Louis Béthune, Anastasiia Filippova +3
As the context size of text processed with an LLM grows, the size of KV caches can outstrip the memory allocated for the original model weights. This impacts LLM throughput negativ…