Stochastic Gradient Descent with Momentum: Analysis and Synthesis via Integral Quadratic Constraints

Dennis Gramlich, Carsten W. Scherer, Christian Ebenbauer

This article applies dynamic integral quadratic constraints (IQCs) to the analysis and synthesis of accelerated stochastic gradient algorithms. We consider composite objective func…