10 citations · 47 across the 27 of their papers we have counts for
1 paper · 1 filter
Jack Cai, Lijie Yang, Vasilis Kontonis +2
Long chain-of-thought decoding is memory-bound: the KV cache of the generated tokens grows linearly with the test-time compute being spent. Evicting cache entries does not fix this…