Seesaw: High-throughput LLM Inference via Model Re-sharding

Qidong Su, Wei Zhao, Xin Li +6

To improve the efficiency of distributed large language model (LLM) inference, various parallelization strategies, such as tensor and pipeline parallelism, have been proposed. Howe…