Uncertainty-gated selection for block-sparse attention

Thomas Rossi

Block-sparse attention scales long-context language models by replacing the O(N^2) softmax with a per-query top-k selection over key blocks. This cutoff is myopic: when the k-th an…