Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models

Alberto Messina, Stefano Scotta

Even when decoding with temperature $T=0$, large language models (LLMs) can produce divergent outputs for identical inputs. Recent work by Thinking Machines Lab highlights implemen…