FineServe: A Fine-Grained Dataset and Characterization of Global LLM Serving Workloads

Tiancheng Zhang, Shaoyuan Huang, Mingyuan Wang +3

Large language models (LLMs) are increasingly deployed as always-on online services, making efficient LLM serving a critical systems challenge. Achieving low latency and high throu…