CoRun: Padding is Simple and Efficient for Deterministic LLM Inference

Shiju Zhao, Jiacheng Yang, Qihang Chen +4

Despite fixed sampling parameters and random seeds, Large Language Model (LLM) inference exhibits output inconsistency, which undermines downstream tasks such as model evaluation a…