Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism

Cong Li, Yuzhe Yang, Xuegui Zheng +7

With the advancement of large language models (LLMs), their context windows have rapidly expanded. To meet diverse demands from varying-length requests in online services, existing…