PSC: Extending Context Window of Large Language Models via Phase Shift Calibration

Wenqiao Zhu, Chao Xu, Lulu Wang +1

Rotary Position Embedding (RoPE) is an efficient position encoding approach and is widely utilized in numerous large language models (LLMs). Recently, a lot of methods have been pu…