2 papers
cs.AI2026
iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD
Yiren Zhao, Guanghui Song, Tianrui Qin +3
Long chain-of-thought reasoning substantially increases KV-cache memory during autoregressive decoding, as every generated token introduces new key and value states and causes the…
cs.CL2025
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization
Guanghui Song, Dongping Liao, Yiren Zhao +3
Transformer models face scalability challenges in causal language modeling (CLM) due to inefficient memory allocation for growing key-value (KV) caches, which strains compute and s…