Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding

Liang Zhao, Xiachong Feng, Xiaocheng Feng +6

Built upon the Transformer, large language models (LLMs) have captured worldwide attention due to their remarkable abilities. Nevertheless, all Transformer-based models including L…