3 papers
cs.LG2026
MirrorLA: Reflecting Feature Map for Vision Linear Attention
Weikang Meng, Liangyu Huo, Yadan Luo +3
Linear attention significantly reduces the computational complexity of Transformers from quadratic to linear, yet it consistently lags behind softmax-based attention in performance…
cs.LG2026
STILL: Selecting Tokens for Intra-Layer Hybrid Attention to Linearize LLMs
Weikang Meng, Liangyu Huo, Yadan Luo +4
Linearizing pretrained large language models (LLMs) primarily relies on intra-layer hybrid attention mechanisms to alleviate the quadratic complexity of standard softmax attention.…
cs.CV2025
PolaFormer: Polarity-aware Linear Attention for Vision Transformers
Weikang Meng, Yadan Luo, Xin Li +2
Linear attention has emerged as a promising alternative to softmax-based attention, leveraging kernelized feature maps to reduce complexity from quadratic to linear in sequence len…