Exploring Context Window of Large Language Models via Decomposed Positional Vectors

Zican Dong, Junyi Li, Xin Men +5

Transformer-based large language models (LLMs) typically have a limited context window, resulting in significant performance degradation when processing text beyond the length of t…