1 paper
Yixing Li, Ruobing Xie, Yudong Zhang +3
Transformers rely on position embedding mechanisms in long context modeling in most cases. Rotary Position Embedding (RoPE) embeds positional information with independent 2D rotati…