CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention

Sayak Dutta

Recurrent delta-rule models keep a fixed-size state matrix S (d_v x d_k) that compresses all past context. The state of the art (GDN-2) gates this update with element-wise matrix e…