Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference

Jiayi Yuan, Hao Li, Xinheng Ding +7

Large Language Models (LLMs) are now integral across various domains and have demonstrated impressive performance. Progress, however, rests on the premise that benchmark scores are…