GTA: Grouped-head latenT Attention

Luoyang Sun, Cheng Deng, Jiwen Jiang +5

Attention mechanisms underpin the success of large language models (LLMs), yet their substantial computational and memory overhead poses challenges for optimizing efficiency and pe…