4 papers
A Tale of Two Error Categories: Exploring Concealed Trade-Offs in the Errors of Automated Judges in Evaluation of Uncertainty Quantifiers
Evgenia Ilia, Wilker Aziz
The wide adoption of LLMs across broad NLG applications heightens the importance of providing users with the means to avert errors and hallucinations. Uncertainty quantification is…
Teaching Language Models to Faithfully Express their Uncertainty
Bryan Eikema, Evgenia Ilia, José G. C. de Souza +2
Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in way…
Variability Need Not Imply Error: The Case of Adequate but Semantically Distinct Responses
Evgenia Ilia, Wilker Aziz
With the broader use of language models (LMs) comes the need to estimate their ability to respond reliably to prompts (e.g., are generated responses likely to be correct?). Uncerta…
Predict the Next Word: Humans exhibit uncertainty in this task and language models _____
Evgenia Ilia, Wilker Aziz
Language models (LMs) are statistical models trained to assign probability to human-generated text. As such, it is reasonable to question whether they approximate linguistic variab…