activity
20232026
most citedCautious Optimizers: Improving Training with One Line of Code

3 citations · 3 across the 5 of their papers we have counts for

collaborators
Showing 2025Show all

1 paper · 1 filter

cs.LG2025

Cautious Weight Decay

Lizhang Chen, Jonathan Li, Kaizhao Liang +6

We introduce Cautious Weight Decay (CWD), a one-line, optimizer-agnostic modification that applies weight decay only to parameter coordinates whose signs align with the optimizer u…