LoHan: Low-Cost High-Performance Framework to Fine-Tune 100B Model on a Consumer GPU

Changyue Liao, Mo Sun, Zihan Yang +5

Nowadays, AI researchers become more and more interested in fine-tuning a pre-trained LLM, whose size has grown to up to over 100B parameters, for their downstream tasks. One appro…