Compressing Context to Enhance Inference Efficiency of Large Language Models

Yucheng Li, Bo Dong, Chenghua Lin +1

Large language models (LLMs) achieved remarkable performance across various tasks. However, they face challenges in managing long documents and extended conversations, due to signi…