Showing stat.MLShow all
3 papers · 1 filter
stat.ML2026
Minimax Optimality of Shampoo for Nonstationary Stochastic Optimization
Abir Sarkar, Nirmalya Mandal, Martin T. Wells
We develop a comprehensive theoretical analysis of Matrix Shampoo for tracking time-varying optima in nonstationary stochastic optimization. We derive finite-horizon high-probabili…
stat.ML2026
Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization
Sharan Sahu, Abir Sarkar, Cameron J. Hogan +1
We provide a theoretical analysis of Adam under non-stationary stochastic objectives, separating two regimes: Euclidean tracking under adaptive strong monotonicity of the Adam-prec…
stat.ML2026
On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
Sharan Sahu, Cameron J. Hogan, Martin T. Wells
In this paper, we provide a comprehensive theoretical analysis of Stochastic Gradient Descent (SGD) and its momentum variants (Polyak Heavy-Ball and Nesterov) for tracking time-var…