3 citations · 5 across the 11 of their papers we have counts for
12 papers
Reprogramming Vision-Language Models via Structured Prompt Reparameterization
Zizhao Li, Chengyi Cai, Mohammed Yaqoob Ansari +3
Visual reprogramming adapts pretrained models to downstream tasks by modifying their input and output interfaces while keeping the backbone fixed. In vision-language models, existi…
CARPRT: Class-Aware Zero-Shot Prompt Reweighting for Black-Box Vision-Language Models
Ruijiang Dong, Zesheng Ye, Jianzhong Qi +4
Pre-trained vision-language models (VLMs) enable zero-shot image classification by computing the similarity score between an image and textual descriptions, typically formed by ins…
Sample-Specific Noise Injection For Diffusion-Based Adversarial Purification
Yuhao Sun, Jiacheng Zhang, Zesheng Ye +2
Diffusion-based purification (DBP) methods aim to remove adversarial noise from the input sample by first injecting Gaussian noise through a forward diffusion process, and then rec…
Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction
Zesheng Ye, Chengyi Cai, Ruijiang Dong +4
As large-scale pre-trained foundation models continue to expand in size and capability, efficiently adapting them to specific downstream tasks has become increasingly critical. Des…
Understanding Model Reprogramming for CLIP via Decoupling Visual Prompts
Chengyi Cai, Zesheng Ye, Lei Feng +2
Model reprogramming adapts pretrained models to downstream tasks by modifying only the input and output spaces. Visual reprogramming (VR) is one instance for vision tasks that adds…
Attribute-based Visual Reprogramming for Vision-Language Models
Chengyi Cai, Zesheng Ye, Lei Feng +2
Visual reprogramming (VR) reuses pre-trained vision models for downstream image classification tasks by adding trainable noise patterns to inputs. When applied to vision-language m…
DualCast: A Model to Disentangle Aperiodic Events from Traffic Series
Xinyu Su, Feng Liu, Yanchuan Chang +3
Traffic forecasting is crucial for transportation systems optimisation. Current models minimise the mean forecasting errors, often favouring periodic events prevalent in the traini…
Bayesian-guided Label Mapping for Visual Reprogramming
Chengyi Cai, Zesheng Ye, Lei Feng +2
Visual reprogramming (VR) leverages the intrinsic capabilities of pretrained vision models by adapting their input or output interfaces to solve downstream tasks whose labels (i.e.…
Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data
Jiahan Zhang, Qi Wei, Feng Liu +1
Fine-tuning vision-language models (VLMs) with abundant unlabeled data recently has attracted increasing attention. Existing methods that resort to the pseudolabeling strategy woul…
Sample-specific Masks for Visual Reprogramming-based Prompting
Chengyi Cai, Zesheng Ye, Lei Feng +2
Visual reprogramming (VR) is a prompting technique that aims to re-purpose a pre-trained model (e.g., a classifier on ImageNet) to target tasks (e.g., medical data prediction) by l…
Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models
Jinhao Li, Haopeng Li, Sarah Erfani +3
It has recently been discovered that using a pre-trained vision-language model (VLM), e.g., CLIP, to align a whole query image with several finer text descriptions generated by a l…
Test-Time Multimodal Backdoor Detection by Contrastive Prompting
Yuwei Niu, Shuo He, Qi Wei +3
While multimodal contrastive learning methods (e.g., CLIP) can achieve impressive zero-shot classification performance, recent research has revealed that these methods are vulnerab…