SystolicAttention: Fusing FlashAttention within a Single Systolic Array

Jiawei Lin, Yuanlong Li, Guokai Chen +1

Transformer models rely heavily on the scaled dot-product attention (SDPA) operation, typically implemented as FlashAttention. Characterized by its frequent interleaving of matrix…