50 citations · 196 across the 45 of their papers we have counts for
7 papers · 1 filter
Towards Compatible Fine-tuning for Vision-Language Model Updates
Zhengbo Wang, Jian Liang, Lijun Sheng +3
So far, efficient fine-tuning has become a popular strategy for enhancing the capabilities of foundation models on downstream tasks by learning plug-and-play modules. However, exis…
Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment
Ziang Yan, Zhilin Li, Yinan He +9
Current multimodal large language models (MLLMs) struggle with fine-grained or precise understanding of visuals although they give comprehensive perception and reasoning in a spect…
DIVE: Subgraph Disagreement for Graph Out-of-Distribution Generalization
Xin Sun, Liang Wang, Qiang Liu +2
This paper addresses the challenge of out-of-distribution (OOD) generalization in graph machine learning, a field rapidly advancing yet grappling with the discrepancy between sourc…
LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
Zhengbo Wang, Jian Liang, Ran He +2
Low-rank adaptation, also known as LoRA, has emerged as a prominent method for parameter-efficient fine-tuning of foundation models. Despite its computational efficiency, LoRA stil…
A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation
Zhengbo Wang, Jian Liang, Lijun Sheng +3
Contrastive Language-Image Pretraining (CLIP) has gained popularity for its remarkable zero-shot capacity. Recent research has focused on developing efficient fine-tuning methods,…
Protecting Model Adaptation from Trojans in the Unlabeled Data
Lijun Sheng, Jian Liang, Ran He +2
Model adaptation tackles the distribution shift problem with a pre-trained model instead of raw data, which has become a popular paradigm due to its great privacy protection. Exist…
Exploring Vacant Classes in Label-Skewed Federated Learning
Kuangpu Guo, Yuhe Ding, Jian Liang +3
Label skews, characterized by disparities in local label distribution across clients, pose a significant challenge in federated learning. As minority classes suffer from worse accu…