SpecAttn: Speculating Sparse Attention

Harsh Shah

Large Language Models (LLMs) face significant computational bottlenecks during inference due to the quadratic complexity of self-attention mechanisms, particularly as context lengt…