Stem: Rethinking Causal Information Flow in Sparse Attention

Lin Niu, Xin Luo, Linchuan Xie +4

The quadratic computational complexity of self-attention remains a fundamental bottleneck for scaling Large Language Models (LLMs) to long contexts, particularly during the pre-fil…