Effectively Compress KV Heads for LLM

Hao Yu, Zelan Yang, Shen Li +2

The advent of pre-trained large language models (LLMs) has revolutionized various natural language processing tasks. These models predominantly employ an auto-regressive decoding m…