2 papers
math.OC2025
Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization
Yufeng Yang, Erin Tripp, Yifan Sun +2
Recent studies have shown that many nonconvex machine learning problems satisfy a generalized-smooth condition that extends beyond traditional smooth nonconvex optimization. Howeve…
stat.ML2025
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
Qi Zhang, Yi Zhou, Shaofeng Zou
This paper provides the first tight convergence analyses for RMSProp and Adam in non-convex optimization under the most relaxed assumptions of coordinate-wise generalized smoothnes…