3 papers
cs.CL2025
LeWiDi-2025 at NLPerspectives: Third Edition of the Learning with Disagreements Shared Task
Elisa Leonardelli, Silvia Casola, Siyao Peng +8
Many researchers have reached the conclusion that AI models should be trained to be aware of the possibility of variation and disagreement in human judgments, and evaluated as per…
stat.ME2025
Understanding Model Calibration -- A gentle introduction and visual exploration of calibration and the expected calibration error (ECE)
Maja Pavlovic
To be considered reliable, a model must be calibrated so that its confidence in each decision closely reflects its true outcome. In this blogpost we'll take a look at the most comm…
cs.CL2024
Understanding The Effect Of Temperature On Alignment With Human Opinions
Maja Pavlovic, Massimo Poesio
With the increasing capabilities of LLMs, recent studies focus on understanding whose opinions are represented by them and how to effectively extract aligned opinion distributions.…