ReAttention: Training-Free Infinite Context with Finite Attention Scope

Xiaoran Liu, Ruixiao Li, Qipeng Guo +7

The long-context capability of the Large Language Models (LLM) has made significant breakthroughs, but the maximum supported context length in length extrapolation remains a critic…