Showing 2026Show all
2 papers · 1 filter
stat.ML2026
Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization
Sharan Sahu, Abir Sarkar, Cameron J. Hogan +1
We provide a theoretical analysis of Adam under non-stationary stochastic objectives, separating two regimes: Euclidean tracking under adaptive strong monotonicity of the Adam-prec…
stat.ML2026
On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
Sharan Sahu, Cameron J. Hogan, Martin T. Wells
In this paper, we provide a comprehensive theoretical analysis of Stochastic Gradient Descent (SGD) and its momentum variants (Polyak Heavy-Ball and Nesterov) for tracking time-var…