Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack

Xiaoyue Xu, Qinyuan Ye, Xiang Ren

We introduce Lifelong ICL, a problem setting that challenges long-context language models (LMs) to learn a sequence of language tasks through in-context learning (ICL). We further…