Logit Attenuating Weight Normalization

Aman Gupta, Rohan Ramanath, Jun Shi +4

Over-parameterized deep networks trained using gradient-based optimizers are a popular choice for solving classification and ranking problems. Without appropriately tuned ${\ell }_2$…