Extending LLMs' Context Window with 100 Samples

Yikai Zhang, Junlong Li, Pengfei Liu

Large Language Models (LLMs) are known to have limited extrapolation ability beyond their pre-trained context window, constraining their application in downstream tasks with length…