Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA

Nuocheng Yang, Sihua Wang, Ouwen Huan +3

Decentralized federated learning (DFL) based on low-rank adaptation (LoRA) enables mobile devices with multi-task datasets to collaboratively fine-tune a large language model (LLM)…