Minimax Optimality of Shampoo for Nonstationary Stochastic Optimization

Abir Sarkar, Nirmalya Mandal, Martin T. Wells

We develop a comprehensive theoretical analysis of Matrix Shampoo for tracking time-varying optima in nonstationary stochastic optimization. We derive finite-horizon high-probabili…