Rectified Sparse Attention

Yutao Sun, Tianzhu Ye, Li Dong +6

Efficient long-sequence generation is a critical challenge for Large Language Models. While recent sparse decoding methods improve efficiency, they suffer from KV cache misalignmen…