74 citations · 162 across the 8 of their papers we have counts for
3 papers · 1 filter
A global analysis of metrics used for measuring performance in natural language processing
Kathrin Blagec, Georg Dorffner, Milad Moradi +2
Measuring the performance of natural language processing models is challenging. Traditionally used metrics, such as BLEU and ROUGE, originally devised for machine translation and s…
Mapping global dynamics of benchmark creation and saturation in artificial intelligence
Simon Ott, Adriano Barbosa-Silva, Kathrin Blagec +2
Benchmarks are crucial to measuring and steering progress in artificial intelligence (AI). However, recent studies raised concerns over the state of AI benchmarking, reporting issu…
Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals
Kathrin Blagec, Jakob Kraiger, Wolfgang Frühwirt +1
Publicly accessible benchmarks that allow for assessing and comparing model performances are important drivers of progress in artificial intelligence (AI). While recent advances in…