The Dual Averaging Power-Prox Method with Application to Heavy-Tail Incremental Gradient

Yuan Gao, Jeremy Rack, Sebastian U. Stich

We study finite-sum composite optimization under two departures from classical stochastic gradient descent theory that are central in practice: incremental gradient access and heav…