Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloads
Jingwei Zuo, Xinze Feng, Zien Liu +5
Low-Rank Adaptation (LoRA) is now the dominant method for parameter-efficient fine-tuning of large language models, but achieving a high-quality adapter often requires systematic h…
cs.LG2024
Empowering Distributed Training with Sparsity-driven Data Synchronization
Zhuang Wang, Zhaozhuo Xu, Jingyi Xi +3
Distributed training is the de facto standard to scale up the training of deep learning models with multiple GPUs. Its performance bottleneck lies in communications for gradient sy…