Top-$n\sigma$: Not All Logits Are You Need

Chenxia Tang, Jianchun Liu, Hongli Xu +1

Large language models (LLMs) typically employ greedy decoding or low-temperature sampling for reasoning tasks, reflecting a perceived trade-off between diversity and accuracy. We c…