${\mu }^2$-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism

Tehila Dahan, Kfir Y. Levy

We consider stochastic convex optimization problems where the objective is an expectation over smooth functions. For this setting we suggest a novel gradient estimate that combines…