Large Language Model as Token Compressor and Decompressor

Wenbing Li, Yiran Wang, Zikai Song +4

In this paper, we study whether an off-the-shelf LLM can be adapted into a discrete, variable-length token compressor and decompressor for long-context processing. To this end, we…