1 paper · 1 filter
Yiren Zhao, Guanghui Song, Tianrui Qin +3
Long chain-of-thought reasoning substantially increases KV-cache memory during autoregressive decoding, as every generated token introduces new key and value states and causes the…