Revisiting Long-context Modeling from Context Denoising Perspective

Zecheng Tang, Baibei Ji, Juntao Li +3

Long-context models (LCMs) have demonstrated great potential in processing long sequences, facilitating many real-world applications. The success of LCMs can be attributed to their…