Debiased Fine-Tuning for Vision-language Models by Prompt Regularization

Beier Zhu, Yulei Niu, Saeil Lee +2

We present a new paradigm for fine-tuning large-scale visionlanguage pre-trained models on downstream task, dubbed Prompt Regularization (ProReg). Different from traditional fine-t…