collaborators

1 paper

cs.LG2022

Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum

Kirby Banman, Liam Peet-Pare, Nidhi Hegde +2

Most convergence guarantees for stochastic gradient descent with momentum (SGDm) rely on iid sampling. Yet, SGDm is often used outside this regime, in settings with temporally corr…