2 papers
cs.LG2025
Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balance
Ao Shen, Qiang Wang, Zhiquan Lai +2
Large Language Models (LLMs) have demonstrated impressive performance across various domains. However, the enormous number of model parameters makes fine-tuning challenging, signif…
cs.DC2024
Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models
Wei Wang, Zhiquan Lai, Shengwei Li +5
The size of deep learning models has been increasing to enhance model quality. The linear increase in training computation budget with model size means that training an extremely l…