Non-Determinism of "Deterministic" LLM Settings

Berk Atil, Sarp Aykent, Alexa Chittams +10

LLM (large language model) practitioners commonly notice that outputs can vary for the same inputs under settings expected to be deterministic. Yet the questions of how pervasive t…