FLoCoRA: Federated learning compression with low-rank adaptation

Lucas Grativol Ribeiro, Mathieu Leonardon, Guillaume Muller +2

Low-Rank Adaptation (LoRA) methods have gained popularity in efficient parameter fine-tuning of models containing hundreds of billions of parameters. In this work, instead, we demo…