Improving Adversarial Robustness of Zero-Shot CLIP with Confidence-Aware Weighting

Nikoo Naghavian, Mostafa Tavassolipour

Vision-language models such as CLIP demonstrate impressive zero-shot generalization but remain highly vulnerable to adversarial attacks. Prior adversarial methods treat all samples…