ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production

Yuxing Xiang, Xue Li, Kun Qian +3

With the widespread adoption of Large Language Models (LLMs), serving LLM inference requests has become an increasingly important task, attracting active research advancements. Pra…