4 papers
6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models
Leon Mayer, Piotr Kalinowski, Caroline Ebersbach +6
Vision-language models (VLMs) are increasingly integrated into clinical workflows. However, existing benchmarks primarily assess performance on common anatomical presentations and…
Current validation practice undermines surgical AI development
Annika Reinke, Ziying O. Li, Minu D. Tizabi +97
Surgical data science (SDS) is rapidly advancing, yet clinical adoption of artificial intelligence (AI) in surgery remains limited, with inadequate validation as an important contr…
Challenging Vision-Language Models with Surgical Data: A New Dataset and Broad Benchmarking Study
Leon Mayer, Tim Rädsch, Dominik Michael +8
While traditional computer vision models have historically struggled to generalize to endoscopic domains, the emergence of foundation models has shown promising cross-domain perfor…
Bridging vision language model (VLM) evaluation gaps with a framework for scalable and cost-effective benchmark generation
Tim Rädsch, Leon Mayer, Simon Pavicic +8
Reliable evaluation of AI models is critical for scientific progress and practical application. While existing VLM benchmarks provide general insights into model capabilities, thei…