CompLLM: Compression for Long Context Q&A

Gabriele Berton, Jayakrishnan Unnikrishnan, Son Tran +1

Large Language Models (LLMs) face significant computational challenges when processing long contexts due to the quadratic complexity of self-attention. While soft context compressi…