FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel

Ran Yan, Youhe Jiang, Zhuoming Chen +3

Recent advances in sparse attention mechanisms have demonstrated strong potential for reducing the computational cost of long-context training and inference in large language model…