28 citations · 48 across the 12 of their papers we have counts for
5 papers · 1 filter
A Sufficient Condition for Convergences of Adam and RMSProp
Fangyu Zou, Li Shen, Zequn Jie +2
Adam and RMSProp are two of the most influential adaptive stochastic algorithms for training deep neural networks, which have been pointed out to be divergent even in the convex se…
A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration
Li Shen, Congliang Chen, Fangyu Zou +3
Integrating adaptive learning rate and momentum techniques into SGD leads to a large class of efficiently accelerated adaptive stochastic algorithms, such as AdaGrad, RMSProp, Adam…
An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method
Li Shen, Peng Sun, Yitong Wang +2
We propose a novel algorithmic framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-gradient (VMOR-HPE) method with a global convergence guarantee for the maximal monoto…
A Generalized Matrix Splitting Algorithm
Ganzhao Yuan, Wei-Shi Zheng, Li Shen +1
Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization, norm reg…
A Decomposition Algorithm for the Sparse Generalized Eigenvalue Problem
Ganzhao Yuan, Li Shen, Wei-Shi Zheng
The sparse generalized eigenvalue problem arises in a number of standard and modern statistical learning models, including sparse principal component analysis, sparse Fisher discri…