BF1: A Causal Dyadic Sparse-Attention Retrofit for Efficient Long-Context Transformers

Hina Dixit

Dense causal attention remains expensive at long context even when implemented with highly optimized exact kernels. We study BF1, a deterministic block-aligned dyadic sparse-attent…