1 paper
Xinhang Zhang, Boning Zhang, Chengchun Liu +4
As sequence lengths continue to grow, distributed attention based on sequence parallel (e.g., Ring-Attention) has become a mainstream solution for large language models (LLMs). Nev…