SPIN: Shadow Predictive Indexer for Sparse Attention

Yao Fu, Jiahan Chang, Cyrus Chang +12

Indexer-based sparse attention reduces the cost of core attention by passing only a fixed, small number of important tokens to it. However, the indexer must still score the entire…