paper

Stochastic Subgradient Descent on a Generic Definable Function Converges to a Minimizer

arXiv:2109.02455

Abstract

It was previously shown by Davis and Drusvyatskiy that every Clarke critical point of a generic, semialgebraic (and more generally definable in an o-minimal structure), weakly convex function is lying on an active manifold and is either a local minimum or an active strict saddle. In the first part of this work, we show that when the weak convexity assumption fails a third type of point appears: a sharply repulsive critical point. Moreover, we show that the corresponding active manifolds satisfy the Verdier and the angle conditions which were introduced by us in our previous work. In the second part of this work, we show that, under a density-like assumption on the perturbation sequence, the stochastic subgradient descent (SGD) avoids sharply repulsive critical points with probability one. We show that such a density-like assumption could be obtained upon adding a small random perturbation (e.g. a nondegenerate Gaussian) at each iteration of the algorithm. These results, combined with our previous work on the avoidance of active strict saddles, show that the SGD on a generic definable (e.g. semialgebraic) function converges to a local minimum.

This paper was withdrawn due to a mistake in the work of Benaïm-Hofbauer-Sorin "Stochastic Approximations and Differential Inclusions". In the latter, the equivalence in Theorem 4.1 is not true and in particular the linearly interpolated process of the iterates is not an APT of the associated DI. This equivalence was at the heart of Propositions 7, 8 and Theorem 2 of the present paper

References in corpus (5)