269 citations · 414 across the 29 of their papers we have counts for
1 paper · 1 filter
Yiwen Chen, Joshua Ainslie, Krzysztof Choromanski +4
Rotary Position Embedding (RoPE) has been widely adopted in transformer-based large language models. However, its log-linear frequency schedule, originally designed to produce long…