MSTGD:A Memory Stochastic sTratified Gradient Descent Method with an Exponential Convergence Rate

Aixiang, Chen, Jinting Zhang +2

The fluctuation effect of gradient expectation and variance caused by parameter update between consecutive iterations is neglected or confusing by current mainstream gradient optim…