Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategy

Chengli Tan, Jiangshe Zhang, Junmin Liu +2

Recently, sharpness-aware minimization (SAM) has attracted much attention because of its surprising effectiveness in improving generalization performance. However, compared to stoc…