A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing

William Nixon, Jon Durbin, Florian Standhartinger +2

Large Language Model (LLM) serving has become a critical cloud workload, and realistic traces are essential for motivating and benchmarking serving systems. However, existing LLM s…