When Prompts Ignore Structure: Graph-Based Attribute Reasoning for Calibrated VLMs

Tanay Sodha, Aditya Sharma, Ramya Hebbalaguppe +2

Reliable confidence estimation remains a key limitation of test-time adaptation in vision-language models (VLMs), where prompt tuning improves zero-shot accuracy but often degrades…