2 citations · 4 across the 7 of their papers we have counts for
3 papers · 1 filter
Black-box Uncertainty Quantification Method for LLM-as-a-Judge
Nico Wagner, Michael Desmond, Rahul Nair +6
LLM-as-a-Judge is a widely used method for evaluating the performance of Large Language Models (LLMs) across various tasks. We address the challenge of quantifying the uncertainty…
Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences
Zahra Ashktorab, Michael Desmond, Qian Pan +7
Evaluation of large language model (LLM) outputs requires users to make critical judgments about the best outputs across various configurations. This process is costly and takes ti…
On Efficient and Statistical Quality Estimation for Data Annotation
Jan-Christoph Klie, Juan Haladjian, Marc Kirchner +1
Annotated datasets are an essential ingredient to train, evaluate, compare and productionalize supervised machine learning models. It is therefore imperative that annotations are o…