Effective Gradient Sample Size via Variation Estimation for Accelerating Sharpness aware Minimization

Jiaxin Deng, Junbiao Pang, Baochang Zhang +1

Sharpness-aware Minimization (SAM) has been proposed recently to improve model generalization ability. However, SAM calculates the gradient twice in each optimization step, thereby…