ICPC: In-context Prompt Compression with Faster Inference

Ziyang Yu, Yuyu Liu

Despite the recent success of Large Language Models (LLMs), it remains challenging to feed LLMs with long prompts due to the fixed size of LLM inputs. As a remedy, prompt compressi…