4 papers
Stochastic versus Deterministic in Stochastic Gradient Descent
Runze Li, Jintao Xu, Wenxun Xing
This paper theoretically reanalyzes the convergence of the mini-batch stochastic gradient descent (SGD) for a structured minimization problem involving a finite-sum function with i…
Depth-first directional search for nonconvex optimization
Yuxuan Zhang, Wenxun Xing
Random search methods are widely used for global optimization due to their theoretical generality and implementation simplicity. This paper proposes a depth-first directional searc…
Stable gradient-adjusted root mean square propagation on least squares problem
Runze Li, Jintao Xu, Wenxun Xing
Root mean square propagation (abbreviated as RMSProp) is a first-order stochastic algorithm used in machine learning widely. In this paper, a stable gradient-adjusted RMSProp (abbr…
ADMM Algorithms for Residual Network Training: Convergence Analysis and Parallel Implementation
Jintao Xu, Yifei Li, Wenxun Xing
We propose both serial and parallel proximal (linearized) alternating direction method of multipliers (ADMM) algorithms for training residual neural networks. In contrast to backpr…