74 citations · 162 across the 8 of their papers we have counts for
8 papers
A global analysis of metrics used for measuring performance in natural language processing
Kathrin Blagec, Georg Dorffner, Milad Moradi +2
Measuring the performance of natural language processing models is challenging. Traditionally used metrics, such as BLEU and ROUGE, originally devised for machine translation and s…
Mapping global dynamics of benchmark creation and saturation in artificial intelligence
Simon Ott, Adriano Barbosa-Silva, Kathrin Blagec +2
Benchmarks are crucial to measuring and steering progress in artificial intelligence (AI). However, recent studies raised concerns over the state of AI benchmarking, reporting issu…
Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals
Kathrin Blagec, Jakob Kraiger, Wolfgang Frühwirt +1
Publicly accessible benchmarks that allow for assessing and comparing model performances are important drivers of progress in artificial intelligence (AI). While recent advances in…
A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks
Kathrin Blagec, Adriano Barbosa-Silva, Simon Ott +1
Research in artificial intelligence (AI) is addressing a growing number of tasks through a rapidly growing number of models and methodologies. This makes it difficult to keep track…
Neural sentence embedding models for semantic similarity estimation in the biomedical domain
Kathrin Blagec, Hong Xu, Asan Agibetov +1
BACKGROUND: In this study, we investigated the efficacy of current state-of-the-art neural sentence embedding models for semantic similarity estimation of sentences from biomedical…
GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain
Milad Moradi, Kathrin Blagec, Florian Haberl +1
Deep neural language models have set new breakthroughs in many tasks of Natural Language Processing (NLP). Recent work has shown that deep transformer language models (pretrained o…
Deep learning models are not robust against noise in clinical text
Milad Moradi, Kathrin Blagec, Matthias Samwald
Artificial Intelligence (AI) systems are attracting increasing interest in the medical domain due to their ability to learn complicated tasks that require human intelligence and ex…
A critical analysis of metrics used for measuring progress in artificial intelligence
Kathrin Blagec, Georg Dorffner, Milad Moradi +1
Comparing model performances on benchmark datasets is an integral part of measuring and driving progress in artificial intelligence. A model's performance on a benchmark dataset is…