Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models

Sunny Gupta, Shambhavi Shanker, Amit Sethi

Federated fine-tuning of foundation models using Low-Rank Adaptation (LoRA) offers a communication efficient solution for distributed learning. However, existing federated LoRA met…