9.4k citations · 11.7k across the 21 of their papers we have counts for
1 paper · 1 filter
Ilya Loshchilov
We note that decoupled weight decay regularization is a particular case of weight norm control where the target norm of weights is set to 0. Any optimization method (e.g., Adam) wh…