1 citations · 1 across the 5 of their papers we have counts for
1 paper · 1 filter
Ranjith Chodavarapu, Lei Xu
KV caching is a ubiquitous optimization in autoregressive transformer inference, long presumed to be numerically equivalent to cache-free computation. This assumption fails under s…