2 papers
cs.LG2026
NormDirection: Restoring the Missing Query Norm in Vision Linear Attention
Weikang Meng, Yadan Luo, Liangyu Huo +4
Linear attention mitigates the quadratic complexity of softmax attention but suffers from a critical loss of expressiveness. We identify two primary causes: (1) The normalization o…
cs.CV2025
PolaFormer: Polarity-aware Linear Attention for Vision Transformers
Weikang Meng, Yadan Luo, Xin Li +2
Linear attention has emerged as a promising alternative to softmax-based attention, leveraging kernelized feature maps to reduce complexity from quadratic to linear in sequence len…