4 papers · 1 filter
Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imaging
Montasir Shams, Chashi Mahiul Islam, Shaeke Salman +2
Vision transformers (ViTs) have rapidly gained prominence in medical imaging tasks such as disease classification, segmentation, and detection due to their superior accuracy compar…
Unaligning Everything: Or Aligning Any Text to Any Image in Multimodal Models
Shaeke Salman, Md Montasir Bin Shams, Xiuwen Liu
Utilizing a shared embedding space, emerging multimodal models exhibit unprecedented zero-shot capabilities. However, the shared embedding space could lead to new vulnerabilities i…
Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models
Shaeke Salman, Md Montasir Bin Shams, Xiuwen Liu +1
Transformer-based models have dominated natural language processing and other areas in the last few years due to their superior (zero-shot) performance on benchmark datasets. Howev…
Intriguing Equivalence Structures of the Embedding Space of Vision Transformers
Shaeke Salman, Md Montasir Bin Shams, Xiuwen Liu
Pre-trained large foundation models play a central role in the recent surge of artificial intelligence, resulting in fine-tuned models with remarkable abilities when measured on be…