Sliding Window Attention Training for Efficient Large Language Models

Zichuan Fu, Wentao Song, Yejing Wang +7

Recent advances in transformer-based Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks. However, their quadratic computational complexity…