5 papers
Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging
Kuangpu Guo, Aijing Yu, Jian Liang +4
Model merging has emerged as a promising paradigm for enabling multi-task capabilities without additional training. However, traditional basic merging methods often experience perf…
Fine-Grained VLM Fine-tuning via Latent Hierarchical Adapter Learning
Yumiao Zhao, Bo Jiang, Yuhe Ding +3
Adapter-based approaches have garnered attention for fine-tuning pre-trained Vision-Language Models (VLMs) on few-shot classification tasks. These methods strive to develop a light…
Harmonizing and Merging Source Models for CLIP-based Domain Generalization
Yuhe Ding, Jian Liang, Bo Jiang +3
CLIP-based domain generalization aims to improve model generalization to unseen domains by leveraging the powerful zero-shot classification capabilities of CLIP and multiple source…
Learning to Rank Pre-trained Vision-Language Models for Downstream Tasks
Yuhe Ding, Bo Jiang, Aihua Zheng +2
Vision language models (VLMs) like CLIP show stellar zero-shot capability on classification benchmarks. However, selecting the VLM with the highest performance on the unlabeled dow…
Exploring Vacant Classes in Label-Skewed Federated Learning
Kuangpu Guo, Yuhe Ding, Jian Liang +3
Label skews, characterized by disparities in local label distribution across clients, pose a significant challenge in federated learning. As minority classes suffer from worse accu…