831 citations · 1.5k across the 9 of their papers we have counts for
6 papers · 1 filter
Medical Imaging AI Competitions Lack Fairness
Annika Reinke, Evangelia Christodoulou, Sthuthi Sadananda +34
Benchmarking competitions are central to the development of artificial intelligence (AI) in medical imaging, defining performance standards and shaping methodological progress. How…
Bridging vision language model (VLM) evaluation gaps with a framework for scalable and cost-effective benchmark generation
Tim Rädsch, Leon Mayer, Simon Pavicic +8
Reliable evaluation of AI models is critical for scientific progress and practical application. While existing VLM benchmarks provide general insights into model capabilities, thei…
Why is the winner the best?
Matthias Eisenmann, Annika Reinke, Vivienn Weru +122
International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to in…
Understanding metric-related pitfalls in image analysis validation
Annika Reinke, Minu D. Tizabi, Michael Baumgartner +75
Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation int…
Labeling instructions matter in biomedical image analysis
Tim Rädsch, Annika Reinke, Vivienn Weru +7
Biomedical image analysis algorithm validation depends on high-quality annotation of reference datasets, for which labeling instructions are key. Despite their importance, their op…
Metrics reloaded: Recommendations for image analysis validation
Lena Maier-Hein, Annika Reinke, Patrick Godau +71
Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen…