LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs

Mohamed Ali Souibgui, Jan Fostier, Rodrigo Abadía-Heredia +3

Transformers are mostly relying on softmax attention, which introduces quadratic complexity with respect to sequence length and remains a major bottleneck for efficient inference.…