3 papers
cs.LG2026
Preserving Updates in Low-Bit Recurrent State Quantization
Haotian Li, Liran Dong, Kaicheng Xiao +1
Linear attention and recurrent models compress past tokens into a fixed-size state that is updated at every step. However, storing these states in high precision incurs substantial…
cs.LG2026
Bridging KV-Cache Quantization and Linear Attention: From Theory to Pretrained Weight Migration
Kaicheng Xiao, Liran Dong, Haotian Li +1
KV-cache quantization and linear attention are two representative approaches to tackling the storage and computational costs of Transformers. KV-cache quantization compresses indiv…
cs.LG2026
RAM-Net: Linear-Time Sequence Modeling with Sparsely Addressable State
Kaicheng Xiao, Haotian Li, Liran Dong +1
Linear attention offers an efficient alternative to full attention with a fixed-size recurrent state. However, this state is shared by all tokens, so information from distinct toke…