ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloads

Jingwei Zuo, Xinze Feng, Zien Liu +5

Low-Rank Adaptation (LoRA) is now the dominant method for parameter-efficient fine-tuning of large language models, but achieving a high-quality adapter often requires systematic h…