FastAttention: Extend FlashAttention2 to NPUs and Low-resource GPUs

Haoran Lin, Xianzhi Yu, Kang Zhao +17

FlashAttention series has been widely applied in the inference of large language models (LLMs). However, FlashAttention series only supports the high-level GPU architectures, e.g.,…