3 papers
math.OC2022
Expected Value of Matrix Quadratic Forms with Wishart distributed Random Matrices
Melinda Hagedorn
To explore the limits of a stochastic gradient method, it may be useful to consider an example consisting of an infinite number of quadratic functions. In this context, it is appro…
math.OC2022
Iteration Complexity of Fixed-Step Methods by Nesterov and Polyak for Convex Quadratic Functions
Melinda Hagedorn, Florian Jarre
This note considers the momentum method by Polyak and the accelerated gradient method by Nesterov, both without line search but with fixed step length applied to strictly convex qu…
math.OC2022
Optimized convergence of stochastic gradient descent by weighted averaging
Melinda Hagedorn, Florian Jarre
Under mild assumptions stochastic gradient methods asymptotically achieve an optimal rate of convergence if the arithmetic mean of all iterates is returned as an approximate optima…