R-LoRA: Randomized Multi-Head LoRA for Efficient Multi-Task Learning

Jinda Liu, Yi Chang, Yuan Wu

Fine-tuning large language models (LLMs) is computationally expensive, and Low-Rank Adaptation (LoRA) provides a cost-effective solution by approximating weight updates through low…