On Evaluating Performance of LLM Inference Serving Systems

Amey Agrawal, Nitin Kedia, Anmol Agarwal +5

The rapid evolution of Large Language Model (LLM) inference systems has yielded significant efficiency improvements. However, our systematic analysis reveals that current evaluatio…