6 citations · 13 across the 6 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
On Surprising Effectiveness of Masking Updates in Adaptive Optimizers
Taejong Joo, Wenhan Xia, Cheolmin Kim +2
Training large language models (LLMs) relies almost exclusively on dense adaptive optimizers with increasingly sophisticated preconditioners. We challenge this by showing that rand…
cs.LG2024★ 5 cited
Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning
Wenhan Xia, Chengwei Qin, Elad Hazan
Fine-tuning is the primary methodology for tailoring pre-trained large language models to specific tasks. As the model's scale and the diversity of tasks expand, parameter-efficien…