When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning

Marko Haralović, Sounic Akkaraju, Carlo Baretta +2

Foundation models for medical image segmentation, like prompt-based MedSAM, generalize well across domains and modalities, often in zero or few-shot setups. However, their performa…