3 citations · 9 across the 5 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures
Yuyang Wu, Yufeng Du, Hao Peng
Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and distinguishing nearby positions. Deter…
cs.CL2026
RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
Yufeng Du, Phillip Harris, Minyang Tian +5
We identify intrinsic limitations of Rotary Positional Embeddings (RoPE) in Transformer-based long-context language models. Our theoretical analysis abstracts away from the specifi…