Optimizing Native Sparse Attention with Latent Attention and Local Global Alternating Strategies

Yuxuan Hu, Jianchao Tan, Jiaqi Zhang +7

In this work, we conduct a systematic analysis of Native Sparse Attention (NSA) and propose targeted improvements that enhance long-context modeling. A key insight is that alternat…