Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences

Vaishnavi Shrivastava, Ananya Kumar, Percy Liang

Language models (LMs) should provide reliable confidence estimates to help users detect mistakes in their outputs and defer to human experts when necessary. Asking a language model…