Enhancing Linear Attention with Residual Learning

Xunhao Lai, Jialiang Kang, Jianqiao Lu +2

Linear attention offers a linear-time alternative to self-attention but often struggles to capture long-range patterns. We revisit linear attention through a prediction-correction…