10 citations · 18 across the 15 of their papers we have counts for
4 papers · 1 filter
LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
Yi Pan, Haocheng Xi, Kan Zhu +10
Recent LLMs increasingly adopt hybrid designs that replace standard attention with linear attention, such as Gated DeltaNet (GDN) and Kimi Delta Attention (KDA). Although they comp…
EasyPPO: Stabilizing the Critic Is Key
Xuanyi Zhou, Qiuyang Mang, Huanzhi Mao +7
A key strength of Proximal Policy Optimization (PPO) is its learned critic, which uses historical trajectories collected during reinforcement learning to estimate expected returns…
Taylor-Calibrate: Principled Initialization for Hybrid Linear Attention Distillation
Zhongzhu Zhou, Qingyang Wu, Junxiong Wang +4
Hybrid linear attention models offer an appealing path to faster long-context inference: they reduce the quadratic cost and KV-cache burden of full softmax attention while retainin…
MRNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling
Mayank Mishra, Shawn Tan, Ion Stoica +3
Transformers are highly parallel but are limited to computations in the TC complexity class, excluding tasks such as entity tracking and code execution that provably require gr…