Semantic Uncertainty Quantification Needs Factual Equivalence

Joseph Hoche, Quentin Guimard, Gianni Franchi

Semantic uncertainty quantification for large language models rests on a common template: sample several answers, measure how much they agree, and treat disagreement as uncertainty…