5 citations · 7 across the 8 of their papers we have counts for
1 paper · 1 filter
Jihao Xin, Tian Lyu, David Keyes +2
Long-context inference in large language models (LLMs) is bottlenecked by the memory and compute of the key-value (KV) cache. Structured pruning is a direct way to shrink it: dropp…