Statistical Modeling and Uncertainty Estimation of LLM Inference Systems

Kaustabha Ray, Nelson Mimura Gonzalez, Bruno Wassermann +2

Large Language Model (LLM) inference systems present significant challenges in statistical performance characterization due to dynamic workload variations, diverse hardware archite…