HSAP: A Hierarchical Sequence-aware Parallelism for Hybrid-Context Generative Models

Songxin Zhang, Zejian Xie, Zhuoyang Song +4

In this paper, we aim to combine the advantages of existing sequence parallelism paradigms and overcomes their drawbacks, the most serious of which is the incapability to correctly…