LLMs are Overconfident: Evaluating Confidence Interval Calibration with FermiEval

Elliot L. Epstein, John Winnicki, Thanawat Sornwanee +1

Large language models (LLMs) excel at numerical estimation but struggle to correctly quantify uncertainty. We study how well LLMs construct confidence intervals around their own an…