Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach

Xinyi Xu, Bingnan Xiao, Shuang Qin +2

Low-rank adaptation (LoRA) represents large language model (LLM) updates with two compact matrix factors, i.e., $A$ and $B$, providing an efficient way to fine-tune large models in…