Acceleration and its Limits in Gaussian Variational Inference

Jaesung R. Park, Uijeong Jang, Yifan Chen

We study whether the classical momentum-based acceleration that improves the condition-number dependence of gradient descent from $\kappa$ to $\sqrt{\kappa }$ for strongly convex, smooth object…