Stationarity Floors and Vanishing Perturbations in Sharpness-Aware Minimization

Abstract

We study a deterministic family of sharpness-aware minimization methods for smooth nonconvex functions. The perturbation is $$ y_k=x_k+ρ\, \frac{\nabla f(x_k)}{\norm{\nabla f(x_k)}^α}, \qquad 0\leqα\leq 1, $$ so that its effective radius is $ρ\norm{\nabla f(x_k)}^{1-α}$. For $0<\alpha \le 1$, we give an explicit complexity bound above the stationarity level $(L\rho {)}^{1/\alpha }$. A one-dimensional quadratic example reaches this level exactly, showing that the bound describes a real limitation of the constant-parameter rule. The unnormalized case $\alpha =0$ is treated separately and requires $L\rho <1$. We then introduce a clipped rule which agrees with the constant-$\rho$ rule away from stationary points and becomes proportional to the gradient near them. The clipped method has $\norm{\nabla f(x_k)}\to0$ and the usual $O(T^{-1/2})$ stationarity bound. Numerical tests on a quadratic function, the Rosenbrock function, and a five-dimensional nonconvex function illustrate the stationarity floor of the unclipped rule and the effect of clipping.