2 papers
cs.LG2026
Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization
Chaewon Moon, Dongkuk Si, Chulhee Yun
We study the implicit bias of Sharpness-Aware Minimization (SAM) when training -layer linear diagonal networks on linearly separable binary classification. For linear models ($L…
cs.LG2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima
Dongkuk Si, Chulhee Yun
Sharpness-Aware Minimization (SAM) is an optimizer that takes a descent step based on the gradient at a perturbation $y_t = x_t + ρ\frac{\nabla f(x_t)}{\lVert \nabla f(x_t) \rVert}…