FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA

Seanie Lee, Sangwoo Park, Dong Bok Lee +5

Low-Rank Adaptation (LoRA), which introduces a product of two trainable low-rank matrices into frozen pre-trained weights, is widely used for efficient fine-tuning of language mode…