PreLort: Prefix-Nested LoRA for Federated Fine-Tuning under Rank Heterogeneity

Muhammad Waseem, Nurbek Tastan, Andrej Jovanovic +4

Federated fine-tuning of large language models using parameter-efficient methods such as LoRA enables privacy-preserving adaptation of foundation models. Heterogeneous hardware res…