Showing 2024Show all
2 papers · 1 filter
cs.CL2024
A Measure of the System Dependence of Automated Metrics
Pius von Däniken, Jan Deriu, Mark Cieliebak
Automated metrics for Machine Translation have made significant progress, with the goal of replacing expensive and time-consuming human evaluations. These metrics are typically ass…
cs.AI2024
Favi-Score: A Measure for Favoritism in Automated Preference Ratings for Generative AI Evaluation
Pius von Däniken, Jan Deriu, Don Tuggener +1
Generative AI systems have become ubiquitous for all kinds of modalities, which makes the issue of the evaluation of such models more pressing. One popular approach is preference r…