ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Models

Manlai Liang, Mandi Liu, Jiangzhou Ji +4

Large Language Models (LLMs) have demonstrated success across many benchmarks. However, they still exhibit limitations in long-context scenarios, primarily due to their short effec…