Rethinking RoPE: A Mathematical Blueprint for N-dimensional Positional Embedding

Haiping Liu, Lijing Lin, Jingyuan Sun +3

Rotary Position Embedding (RoPE) is widely adopted in large language models (LLMs) due to its efficient encoding of relative positions with strong extrapolation capabilities. Howev…