activity
20222026
most citedSparse Matrix in Large Language Model Fine-tuning

2 citations · 3 across the 8 of their papers we have counts for

collaborators
Showing 2024Show all

2 papers · 1 filter

cs.CL2024★ 2 cited

Sparse Matrix in Large Language Model Fine-tuning

Haoze He, Juncheng Billy Li, Xuan Jiang +1

LoRA and its variants have become popular parameter-efficient fine-tuning (PEFT) methods due to their ability to avoid excessive computational costs. However, an accuracy gap often…

cs.LG2024

Adjacent Leader Decentralized Stochastic Gradient Descent

Haoze He, Jing Wang, Anna Choromanska

This work focuses on the decentralized deep learning optimization framework. We propose Adjacent Leader Decentralized Gradient Descent (AL-DSGD), for improving final model performa…