Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference

Quantong Qiu, Zhiyi Hong, Yi Yang +5

The quadratic computational complexity of standard attention mechanisms presents a severe scalability bottleneck for LLMs in long-context scenarios. While hybrid attention mechanis…