SEAL: Scaling to Emphasize Attention for Long-Context Retrieval

Changhun Lee, Minsang Seok, Jun-gyu Jin +2

While many advanced LLMs are designed to handle long sequence data, we can still observe notable quality degradation even within the sequence limit. In this work, we introduce a no…