1 citations · 1 across the 2 of their papers we have counts for
2 papers
cs.LG2025
Cautious Weight Decay
Lizhang Chen, Jonathan Li, Kaizhao Liang +6
We introduce Cautious Weight Decay (CWD), a one-line, optimizer-agnostic modification that applies weight decay only to parameter coordinates whose signs align with the optimizer u…
cs.CL2020★ 1 cited
Aligning the Pretraining and Finetuning Objectives of Language Models
Nuo Wang Pierse, Jingwen Lu
We demonstrate that explicitly aligning the pretraining objectives to the finetuning objectives in language model training significantly improves the finetuning task performance an…