Attention Basin: Why Contextual Position Matters in Large Language Models

Zihao Yi, Delong Zeng, Zhenqing Ling +6

The performance of Large Language Models (LLMs) is significantly sensitive to the contextual position of information in the input. To investigate the mechanism behind this position…