7 citations · 7 across the 6 of their papers we have counts for
4 papers · 1 filter
Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
Ziyu Zhao, Tao Shen, Didi Zhu +5
Low-Rank Adaptation (LoRA) has emerged as a popular technique for fine-tuning large language models (LLMs) to various domains due to its modular design and widespread availability…
OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning
Rui Ye, Wenhao Wang, Jingyi Chai +6
Trained on massive publicly available data, large language models (LLMs) have demonstrated tremendous success across various fields. While more data contributes to better performan…
FedGuCci: Making Local Models More Connected in Landscape for Federated Learning
Zexi Li, Jie Lin, Zhiqi Li +5
Federated learning (FL) involves multiple heterogeneous clients collaboratively training a global model via iterative local updates and model fusion. The generalization of FL's glo…
Improving Model Fusion by Training-time Neuron Alignment with Fixed Neuron Anchors
Zexi Li, Zhiqi Li, Jie Lin +5
Model fusion aims to integrate several deep neural network (DNN) models' knowledge into one by fusing parameters, and it has promising applications, such as improving the generaliz…