19 citations · 47 across the 12 of their papers we have counts for
1 paper · 1 filter
Zuxiong Tan, Will Wei-Jen Wang, Wei Shao +3
Autoregressive large language model (LLM) decoding re-reads a growing key-value (KV) cache at every step, so long-context attention is bound by graphics processing unit (GPU) memor…