1.3k citations · 2.6k across the 33 of their papers we have counts for
35 papers
HeiCo-FOCUS: A Clinically Grounded Dataset for Long-Context Video Understanding
Leon Mayer, Lucas Luttner, Patrick Godau +42
Recent advances in Vision-Language Models (VLMs) have led to rapid progress in video understanding across a wide range of benchmark tasks. However, existing evaluations largely foc…
nnFoundation: 3D Foundation Models for Radiology
Constantin Ulrich Harsy, Tassilo Wald, Karol Gotkowski +80
Radiological artificial intelligence has advanced rapidly, yet most systems remain narrowly task-specific, data-intensive, and fragile under domain shift. Foundation models promise…
Performance uncertainty in medical image analysis: a large-scale investigation of confidence intervals
Pascaline André, Charles Heitz, Evangelia Christodoulou +10
Performance uncertainty quantification is essential for reliable validation and eventual clinical translation of medical imaging artificial intelligence (AI). Confidence intervals…
Medical Imaging AI Competitions Lack Fairness
Annika Reinke, Evangelia Christodoulou, Sthuthi Sadananda +34
Benchmarking competitions are central to the development of artificial intelligence (AI) in medical imaging, defining performance standards and shaping methodological progress. How…
6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models
Leon Mayer, Piotr Kalinowski, Caroline Ebersbach +6
Vision-language models (VLMs) are increasingly integrated into clinical workflows. However, existing benchmarks primarily assess performance on common anatomical presentations and…
Current validation practice undermines surgical AI development
Annika Reinke, Ziying O. Li, Minu D. Tizabi +98
Surgical data science (SDS) is rapidly advancing, yet clinical adoption of artificial intelligence (AI) in surgery remains limited, with inadequate validation as an important contr…
Federated Learning for Surgical Vision in Appendicitis Classification: Results of the FedSurg EndoVis 2024 Challenge
Max Kirchner, Hanna Hoffmann, Alexander C. Jenke +16
Developing generalizable surgical AI requires multi-institutional data, yet privacy constraints preclude direct data sharing, making Federated Learning (FL) a natural candidate. It…
Challenging Vision-Language Models with Surgical Data: A New Dataset and Broad Benchmarking Study
Leon Mayer, Tim Rädsch, Dominik Michael +8
While traditional computer vision models have historically struggled to generalize to endoscopic domains, the emergence of foundation models has shown promising cross-domain perfor…
False Promises in Medical Imaging AI? Assessing Validity of Outperformance Claims
Evangelia Christodoulou, Annika Reinke, Pascaline Andrè +23
Performance comparisons are fundamental in medical imaging Artificial Intelligence (AI) research, often driving claims of superiority based on relative improvements in common perfo…
Bridging vision language model (VLM) evaluation gaps with a framework for scalable and cost-effective benchmark generation
Tim Rädsch, Leon Mayer, Simon Pavicic +8
Reliable evaluation of AI models is critical for scientific progress and practical application. While existing VLM benchmarks provide general insights into model capabilities, thei…
Confidence intervals uncovered: Are we ready for real-world medical imaging AI?
Evangelia Christodoulou, Annika Reinke, Rola Houhou +19
Medical imaging is spearheading the AI transformation of healthcare. Performance reporting is key to determine which methods should be translated into clinical practice. Frequently…
Quality Assured: Rethinking Annotation Strategies in Imaging AI
Tim Rädsch, Annika Reinke, Vivienn Weru +5
This paper does not describe a novel method. Instead, it studies an essential foundation for reliable benchmarking and ultimately real-world application of AI-based image analysis:…
FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures
Lisa Mais, Peter Hirsch, Claire Managan +6
Instance segmentation of neurons in volumetric light microscopy images of nervous systems enables groundbreaking research in neuroscience by facilitating joint functional and morph…
Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps
Florian Kofler, Hendrik Möller, Josef A. Buchner +18
This paper introduces panoptica, a versatile and performance-optimized package designed for computing instance-wise segmentation quality metrics from 2D and 3D segmentation maps. p…
Application-driven Validation of Posteriors in Inverse Problems
Tim J. Adler, Jan-Hinrich Nölke, Annika Reinke +8
Current deep learning-based solutions for image analysis tasks are commonly incapable of handling problems to which multiple different plausible solutions exist. In response, poste…
Challenge Results Are Not Reproducible
Annika Reinke, Georg Grab, Lena Maier-Hein
While clinical trials are the state-of-the-art methods to assess the effect of new medication in a comparative manner, benchmarking in the field of medical image analysis is perfor…
Why is the winner the best?
Matthias Eisenmann, Annika Reinke, Vivienn Weru +122
International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to in…
Deployment of Image Analysis Algorithms under Prevalence Shifts
Patrick Godau, Piotr Kalinowski, Evangelia Christodoulou +5
Domain gaps are among the most relevant roadblocks in the clinical translation of machine learning (ML)-based solutions for medical image analysis. While current research focuses o…
Understanding metric-related pitfalls in image analysis validation
Annika Reinke, Minu D. Tizabi, Michael Baumgartner +75
Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation int…
Biomedical image analysis competitions: The state of current participation practice
Matthias Eisenmann, Annika Reinke, Vivienn Weru +352
The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known abou…
Sources of performance variability in deep learning-based polyp detection
Thuy Nuong Tran, Tim Adler, Amine Yamlahi +8
Validation metrics are a key prerequisite for the reliable tracking of scientific progress and for deciding on the potential clinical translation of methods. While recent initiativ…
Labeling instructions matter in biomedical image analysis
Tim Rädsch, Annika Reinke, Vivienn Weru +7
Biomedical image analysis algorithm validation depends on high-quality annotation of reference datasets, for which labeling instructions are key. Despite their importance, their op…
Metrics reloaded: Recommendations for image analysis validation
Lena Maier-Hein, Annika Reinke, Patrick Godau +71
Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen…
Ten years of image analysis and machine learning competitions in dementia
Esther E. Bron, Stefan Klein, Annika Reinke +4
Machine learning methods exploiting multi-parametric biomarkers, especially based on neuroimaging, have huge potential to improve early diagnosis of dementia and to predict which i…