26 citations · 27 across the 9 of their papers we have counts for
4 papers · 1 filter
Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach
Xinyi Xu, Bingnan Xiao, Shuang Qin +2
Low-rank adaptation (LoRA) represents large language model (LLM) updates with two compact matrix factors, i.e., and , providing an efficient way to fine-tune large models in…
FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning
Bingnan Xiao, Yuan Gao, Bingcong Li +3
Sharpness-aware minimization (SAM) is an effective method for improving the generalization of federated learning (FL) by steering local training toward flat minima. Under data hete…
Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
Nuocheng Yang, Sihua Wang, Ouwen Huan +3
Decentralized federated learning (DFL) based on low-rank adaptation (LoRA) enables mobile devices with multi-task datasets to collaboratively fine-tune a large language model (LLM)…
Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Framework
Shuai Wang, Yanqing Xu, Zhiguo Wang +3
As a novel distributed learning paradigm, federated learning (FL) faces serious challenges in dealing with massive clients with heterogeneous data distribution and computation and…