most citedContext Length Alone Hurts LLM Performance Despite Perfect Retrieval

3 citations · 3 across the 5 of their papers we have counts for

collaborators
Showing cs.LGShow all

1 paper · 1 filter

cs.LG2026

RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures

Yuyang Wu, Yufeng Du, Hao Peng

Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and distinguishing nearby positions. Deter…