Aggregating Low Rank Adapters in Federated Fine-tuning

Evelyn Trautmann, Ian Hales, Martin F. Volk

Fine-tuning large language models requires high computational and memory resources, and is therefore associated with significant costs. When training on federated datasets, an incr…