When LLMs get significantly worse: A statistical approach to detect model degradations

Jonas Kübler, Kailash Budhathoki, Matthäus Kleindessner +4

Minimizing the inference cost and latency of foundation models has become a crucial area of research. Optimization approaches include theoretically lossless methods and others with…