KV Shifting Attention Enhances Language Modeling

Mingyu Xu, Wei Cheng, Bingning Wang +1

The current large language models are mainly based on decode-only structure transformers, which have great in-context learning (ICL) capabilities. It is generally believed that the…