Residual Vector-based Reconstruction as Long-Context Recall Regardless of Context Window Size

MyungHoon Ryu, XinYu Piao, Jong-Kook Kim

Large language models (LLMs) process long contexts, including long documents and lengthy conversations, but face token-level memory usage that increases proportionally to input len…