One Temperature to Rule Them All?

Aviv Orly, Ori Shem-Ur, Yaron Oz

Inference-time sampling is a central mechanism for improving large language models, with temperature controlling the tradeoff between reliability and diversity. While recent work s…