LLM-Pilot: Characterize and Optimize Performance of your LLM Inference Services

Małgorzata Łazuka, Andreea Anghel, Thomas Parnell

As Large Language Models (LLMs) are rapidly growing in popularity, LLM inference services must be able to serve requests from thousands of users while satisfying performance requir…