Learning What Not to Learn: Adversarial Disentangled Prompt Tuning for Robust Vision-Language Models

Yang Chen, Zhan Zhuang, Yanbin Wei +3

While adversarial prompt tuning can enhance robustness of vision-language models efficiently, we find that existing methods aggravate robust generalization overfitting on seen clas…