StateFlow: Sequence Pipeline Parallelism for Long-Context Modeling with Linear Recurrence

Wenxuan Zhao, Yingfa Chen, Xu Han +7

Long-context training is increasingly important for large language models, and linear attention and state space models have become popular for improving long-context efficiency. Ho…