Distance between Relevant Information Pieces Causes Bias in Long-Context LLMs

Runchu Tian, Yanghao Li, Yuepeng Fu +10

Positional bias in large language models (LLMs) hinders their ability to effectively process long inputs. A prominent example is the "lost in the middle" phenomenon, where LLMs str…