3 papers
math.OC2025
Stochastic versus Deterministic in Stochastic Gradient Descent
Runze Li, Jintao Xu, Wenxun Xing
This paper theoretically reanalyzes the convergence of the mini-batch stochastic gradient descent (SGD) for a structured minimization problem involving a finite-sum function with i…
math.OC2024
Stable gradient-adjusted root mean square propagation on least squares problem
Runze Li, Jintao Xu, Wenxun Xing
Root mean square propagation (abbreviated as RMSProp) is a first-order stochastic algorithm used in machine learning widely. In this paper, a stable gradient-adjusted RMSProp (abbr…
cs.LG2024
Your Network May Need to Be Rewritten: Network Adversarial Based on High-Dimensional Function Graph Decomposition
Xiaoyan Su, Yinghao Zhu, Run Li
In the past, research on a single low dimensional activation function in networks has led to internal covariate shift and gradient deviation problems. A relatively small research a…