ADF-LoRA: Alternating Low-Rank Aggregation for Decentralized Federated Fine-Tuning

Xiaoyu Wang, Xiaotian Li, Zhixiang Zhou +2

This paper revisits alternating low-rank updates for federated fine-tuning and examines their behavior in decentralized federated learning (DFL). While alternating the LoRA matrice…