Gated Slot Attention-2: Two-Sided Associative Memory Correction in Linear Attention

Ruijie Li, Shengnan Ding, Weimin Zhang +6

Linear attention models have emerged as efficient alternatives to standard attention, but effectively managing their fixed-size recurrent memory remains challenging. To improve mem…