10 papers
FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuning
Jieming Bian, Lei Wang, Letian Zhang +1
Federated Learning (FL) with Low-Rank Adaptation (LoRA) has become a standard for privacy-preserving LLM fine-tuning. However, existing personalized methods predominantly operated…
Null-Space Filtering for Data-Free Continual Model Merging: Preserving Stability, Promoting Plasticity
Zihuan Qiu, Lei Wang, Yang Cao +7
Data-free continual model merging (DFCMM) aims to fuse independently fine-tuned models into a single backbone that evolves with incoming tasks without accessing task data. This pap…
FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA
Jieming Bian, Lei Wang, Letian Zhang +1
Fine-tuning large language models (LLMs) in federated settings enables privacy-preserving adaptation but suffers from cross-client interference due to model aggregation. Existing f…
LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinement
Jieming Bian, Lei Wang, Letian Zhang +1
Foundation models (FMs) achieve strong performance across diverse tasks with task-specific fine-tuning, yet full parameter fine-tuning is often computationally prohibitive for larg…
Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
Lei Wang, Jieming Bian, Letian Zhang +1
Large Language Models (LLMs) have demonstrated impressive capabilities across various tasks, but fine-tuning them for domain-specific applications often requires substantial domain…
FedEL: Federated Elastic Learning for Heterogeneous Devices
Letian Zhang, Bo Chen, Jieming Bian +2
Federated learning (FL) enables distributed devices to collaboratively train machine learning models while maintaining data privacy. However, the heterogeneous hardware capabilitie…