RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation

Jun Liu, Zhenglun Kong, Peiyan Dong +10

Fine-tuning helps large language models (LLM) recover degraded information and enhance task performance. Although Low-Rank Adaptation (LoRA) is widely used and effective for fine-t…