Measuring and Aligning Abstraction in Vision-Language Models with Medical Taxonomies

Ben Schaper, Maxime Di Folco, Bernhard Kainz +2

Vision-Language Models show strong zero-shot performance for chest X-ray classification, but standard flat metrics fail to distinguish between clinically minor and severe errors. T…