The Query Knows What to Forget: A Second Erase Direction for Linear Attention

Dhruman Gupta, Aritra Das, Debayan Gupta

Linear attention keeps a state of fixed size. At long context, many stored items share this state, and interference between them degrades retrieval. Gated DeltaNet-2 (GDN-2), like…