QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention

Sehyeon Oh, Yongin Kwon, Jemin Lee

FlashAttention improves efficiency through tiling, but its online softmax still relies on floating-point arithmetic for numerical stability, making full quantization difficult. We…