Adaptive Selection of LoRA Components in Privacy-Preserving Federated Learning

Myoungjun Kim, Sangwoo Park, Yoseob Han +1

Differentially private federated fine-tuning of large models with LoRA suffers from aggregation error caused by LoRA's multiplicative structure, which is further amplified by DP no…