Trading-off variance and complexity in stochastic gradient descent

Vatsal Shah, Megasthenis Asteris, Anastasios Kyrillidis +1

Stochastic gradient descent is the method of choice for large-scale machine learning problems, by virtue of its light complexity per iteration. However, it lags behind its non-stoc…