Cross-layer Attention Sharing for Pre-trained Large Language Models

Yongyu Mu, Yuzhang Wu, Yuchun Fan +9

To enhance the efficiency of the attention mechanism within large language models (LLMs), previous works primarily compress the KV cache or group attention heads, while largely ove…