Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures

Liam Cooper, Shinnung Jeong, Hyeran Jeon +2

Large language model (LLM) outputs are expected to be reproducible under greedy decoding, yet in practice the same model, prompt, and software stack produce different outputs on di…