LLoCO: Learning Long Contexts Offline

Sijun Tan, Xiuyu Li, Shishir Patil +5

Processing long contexts remains a challenge for large language models (LLMs) due to the quadratic computational and memory overhead of the self-attention mechanism and the substan…