An Adaptive Proximal Framework for Stochastic Weakly Convex Optimization

Abstract

Many nonsmooth, nonconvex objectives in learning and signal recovery are $\rho$-weakly convex. We minimize such a function when the weak-convexity parameter $\rho$ is unknown and only stochastic candidate and function-difference information is available. We propose the Adaptive Prox-Guided Scheme (APS), a single-trial framework that adapts the proximal parameter online and bidirectionally through a noisy descent test. APS attains a high-probability $O({\epsilon }^{-2})$ iteration bound for Moreau-envelope stationarity. The result allows biased, heavy-tailed function-difference estimates, while the candidate oracle need only be sufficiently accurate with constant probability when the proximal parameter is small and may be arbitrary otherwise. APS does not need to identify which case applies. A model-based extension covers adaptive proximal-point, prox-linear, and proximal-gradient methods. When the oracles are exact, the same analysis yields a deterministic $O({\epsilon }^{-2})$ bound and, for the proximal-point instance, an $\epsilon$-subgradient stationary point.