A sequential smoothing majorant stochastic approximation method for nonconvex nonconcave minimax problems

Abstract

We propose a sequential smoothing majorant stochastic approximation (SMSA) method for nonconvex-nonconcave minimax optimization problems. To overcome the nonconcavity of the inner maximization problem, we introduce a new majorant stochastic approximation with an $O\left({\beta }_N^2\right)$ accuracy bound, where ${\beta }_N$ is the sample coverage radius. The accuracy bound significantly improves upon the $O\left({\beta }_N\right)$ approximation bound for the standard stochastic approximation. Moreover, we establish nonasymptotic bounds for both global optimal values and minimizer sets, and prove consistency for the Clarke stationary points, as ${\beta }_N\downarrow 0$ almost surely. We show that the generated sequence by the SMSA method is bounded, and that the returned point is an approximate Clarke stationary point of the majorant stochastic approximation models. Numerical experiments on a synthetic toy example and robust logistic regression on two UCI datasets demonstrate improved approximation fidelity and lower mean robust test losses relative to the standard sampled approximation.