34 citations · 82 across the 8 of their papers we have counts for
1 paper · 1 filter
Shashank Rajput, Ying Sheng, Sean Owen +1
The size of the key-value (KV) cache plays a critical role in determining both the maximum context length and the number of concurrent requests supported during inference in modern…