LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization

Yi Pan, Haocheng Xi, Kan Zhu +10

Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Attention (KDA). Although they comp…