10 citations · 19 across the 11 of their papers we have counts for
11 papers
On convergence rates of subgradient descent on semialgebraic functions
Evgenii Chzhen, Sholom Schechtman
We analyze the constant step size subgradient method on nonsmooth, nonconvex functions. We identify geometric assumptions on the objective function under which i) its domain admits…
The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networks
Sholom Schechtman, Nicolas Schreuder
We analyze the implicit bias of constant step stochastic subgradient descent (SGD). We consider the setting of binary classification with homogeneous neural networks - a large clas…
The gradient's limit of a definable family of functions admits a variational stratification
Sholom Schechtman
It is well-known that the convergence of a family of smooth functions does not imply the convergence of its gradients. In this work, we show that if the family is definable in an o…
SignSVRG: fixing SignSGD via variance reduction
Evgenii Chzhen, Sholom Schechtman
We consider the problem of unconstrained minimization of finite sums of functions. We propose a simple, yet, practical way to incorporate variance reduction techniques into SignSGD…
Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold
Sholom Schechtman, Daniil Tiapkin, Michael Muehlebach +1
We consider the problem of minimizing a non-convex function over a smooth manifold . We propose a novel algorithm, the Orthogonal Directions Constrained Gradient Metho…
AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks
Louis Leconte, Sholom Schechtman, Eric Moulines
In this paper, we develop a new algorithm, Annealed Skewed SGD - AskewSGD - for training deep neural networks (DNNs) with quantized weights. First, we formulate the training of qua…
Stochastic Subgradient Descent on a Generic Definable Function Converges to a Minimizer
Sholom Schechtman
It was previously shown by Davis and Drusvyatskiy that every Clarke critical point of a generic, semialgebraic (and more generally definable in an o-minimal structure), weakly conv…
Stochastic Subgradient Descent Escapes Active Strict Saddles on Weakly Convex Functions
Pascal Bianchi, Walid Hachem, Sholom Schechtman
In non-smooth stochastic optimization, we establish the non-convergence of the stochastic subgradient descent (SGD) to the critical points recently called active strict saddles by…
Stochastic proximal subgradient descent oscillates in the vicinity of its accumulation set
Sholom Schechtman
We analyze the stochastic proximal subgradient descent in the case where the objective functions are path differentiable and verify a Sard-type condition. While the accumulation se…
Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance
A. Barakat, P. Bianchi, W. Hachem +1
In this paper, a general stochastic optimization procedure is studied, unifying several variants of the stochastic gradient descent such as, among others, the stochastic heavy ball…
Convergence of constant step stochastic gradient descent for non-smooth non-convex functions
Pascal Bianchi, Walid Hachem, Sholom Schechtman
This paper studies the asymptotic behavior of the constant step Stochastic Gradient Descent for the minimization of an unknown function F , defined as the expectation of a non conv…