Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling

Changze Lv, Zhenghua Wang, Yiran Ding +9

Large Language Models (LLMs) still struggle with the ``lost-in-the-middle'' problem, where critical information located in the middle of long-context inputs is often underrepresent…