STS: Efficient Sparse Attention with Speculative Token Sparsity

Jiangnan Yu, Ceyu Xu, Yongji Wu +1

The quadratic complexity of attention imposes severe memory and computational bottlenecks on Large Language Model (LLM) inference. This challenge is particularly acute for emerging…