Quality Text, Robust Vision: The Role of Language in Enhancing Visual Robustness of Vision-Language Models

Futa Waseda, Saku Sugawara, Isao Echizen

Defending pre-trained vision-language models (VLMs), such as CLIP, against adversarial attacks is crucial, as these models are widely used in diverse zero-shot tasks, including ima…