8 citations · 16 across the 4 of their papers we have counts for
4 papers · 1 filter
Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions
Bohan Wang, Huishuai Zhang, Zhi-Ming Ma +1
We provide a simple convergence proof for AdaGrad optimizing non-convex objectives under only affine noise variance and bounded smoothness assumptions. The proof is essentially bas…
Provable Adaptivity of Adam under Non-uniform Smoothness
Bohan Wang, Yushun Zhang, Huishuai Zhang +6
Adam is widely adopted in practical applications due to its fast convergence. However, its theoretical analysis is still far from satisfactory. Existing convergence analyses for Ad…
Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs
Peiyan Hu, Qi Meng, Bingguang Chen +6
Stochastic partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. Neural Operators, generation…
Does Momentum Change the Implicit Regularization on Separable Data?
Bohan Wang, Qi Meng, Huishuai Zhang +4
The momentum acceleration technique is widely adopted in many optimization algorithms. However, there is no theoretical answer on how the momentum affects the generalization perfor…