Attention Instruction: Amplifying Attention in the Middle via Prompting

Meiru Zhang, Zaiqiao Meng, Nigel Collier

The context window of large language models has been extended to 128k tokens or more. However, language models still suffer from position bias and have difficulty in accessing and…