68 citations · 102 across the 22 of their papers we have counts for
6 papers · 1 filter
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Anke Tang, Xianglin Luo, Li Shen +5
Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct…
Federated Learning with Manifold Regularization and Normalized Update Reaggregation
Xuming An, Li Shen, Han Hu +1
Federated Learning (FL) is an emerging collaborative machine learning framework where multiple clients train the global model without sharing their own datasets. In FL, the model i…
Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models
Miaoxi Zhu, Qihuang Zhong, Li Shen +4
Quantization is a promising approach for reducing memory overhead and accelerating inference, especially in large pre-trained language model (PLM) scenarios. While having no access…
Learning from models beyond fine-tuning
Hongling Zheng, Li Shen, Anke Tang +5
Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primaril…
Parameter Efficient Multi-task Model Fusion with Partial Linearization
Anke Tang, Li Shen, Yong Luo +5
Large pre-trained models have enabled significant advances in machine learning and served as foundation components. Model fusion methods, such as task arithmetic, have been proven…
Deep Model Fusion: A Survey
Weishi Li, Yong Peng, Miao Zhang +3
Deep model fusion/merging is an emerging technique that merges the parameters or predictions of multiple deep learning models into a single one. It combines the abilities of differ…