Reading Calibrated Uncertainty from Language Model Trajectories

Aliai Eusebi, Alexander Herzog, Xiaoyu Liang +3

The maximum softmax probability (MSP) represents a default approach when evaluating uncertainty quantification for language model generation with structured output. Although cheap,…