5 papers
Composition, Not Conversation: VLMs Lose the Scene, Not the Thread
L. D. M. S. Sai Teja, Ufaq Khan, N. Siva Gopala Krishna +5
Vision-language models (VLMs) increasingly reason over visual evidence that is cropped, segmented, retrieved, or revealed over time. Yet most VQA benchmarks present the complete im…
MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models
Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan +6
Reliable evaluation of vision-language models (VLMs) and medical vision-language models (Medical-VLMs) requires calibrated confidence, particularly under realistic clinical conditi…
Towards Calibrating Prompt Tuning of Vision-Language Models
Ashshak Sharifdeen, Fahad Shamshad, Muhammad Akhtar Munir +6
Prompt tuning of large-scale vision-language models such as CLIP enables efficient task adaptation without updating model weights. However, it often leads to poor confidence calibr…
Calibration-Aware Prompt Learning for Medical Vision-Language Models
Abhishek Basu, Fahad Shamshad, Ashshak Sharifdeen +2
Medical Vision-Language Models (Med-VLMs) have demonstrated remarkable performance across diverse medical imaging tasks by leveraging large-scale image-text pretraining. However, t…
O-TPT: Orthogonality Constraints for Calibrating Test-time Prompt Tuning in Vision-Language Models
Ashshak Sharifdeen, Muhammad Akhtar Munir, Sanoojan Baliah +2
Test-time prompt tuning for vision-language models (VLMs) is getting attention because of their ability to learn with unlabeled data without fine-tuning. Although test-time prompt…