INT-FlashAttention: Enabling Flash Attention for INT8 Quantization

Shimao Chen, Zirui Liu, Zhiying Wu +6

As the foundation of large language models (LLMs), self-attention module faces the challenge of quadratic time and memory complexity with respect to sequence length. FlashAttention…