Calibration as a First-Class Criterion in LLM Evaluation

Mario Sanz-Guerrero, Katharina von der Wense

Calibration of language models -- the alignment between expressed or implicit confidence and empirical correctness -- is a well-studied subfield within NLP. Methods to measure it a…