28 citations · 48 across the 12 of their papers we have counts for
12 papers
Continual Learning From a Stream of APIs
Enneng Yang, Zhenyi Wang, Li Shen +5
Continual learning (CL) aims to learn new tasks without forgetting previous tasks. However, existing CL methods require a large amount of raw data, which is often unavailable due t…
Efficient-Adam: Communication-Efficient Distributed Adam
Congliang Chen, Li Shen, Wei Liu +1
Distributed adaptive stochastic gradient methods have been widely used for large-scale nonconvex optimization, such as training deep learning models. However, their communication c…
Provably Efficient Convergence of Primal-Dual Actor-Critic with Nonlinear Function Approximation
Jing Dong, Li Shen, Yinggan Xu +1
We study the convergence of the actor-critic algorithm with nonlinear function approximation under a nonconvex-nonconcave primal-dual formulation. Stochastic gradient descent ascen…
A Sufficient Condition for Convergences of Adam and RMSProp
Fangyu Zou, Li Shen, Zequn Jie +2
Adam and RMSProp are two of the most influential adaptive stochastic algorithms for training deep neural networks, which have been pointed out to be divergent even in the convex se…
A Unified Analysis of AdaGrad with Weighted Aggregation and Momentum Acceleration
Li Shen, Congliang Chen, Fangyu Zou +3
Integrating adaptive learning rate and momentum techniques into SGD leads to a large class of efficiently accelerated adaptive stochastic algorithms, such as AdaGrad, RMSProp, Adam…
An Algorithmic Framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-Gradient Method
Li Shen, Peng Sun, Yitong Wang +2
We propose a novel algorithmic framework of Variable Metric Over-Relaxed Hybrid Proximal Extra-gradient (VMOR-HPE) method with a global convergence guarantee for the maximal monoto…
A Generalized Matrix Splitting Algorithm
Ganzhao Yuan, Wei-Shi Zheng, Li Shen +1
Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization, norm reg…
A Decomposition Algorithm for the Sparse Generalized Eigenvalue Problem
Ganzhao Yuan, Li Shen, Wei-Shi Zheng
The sparse generalized eigenvalue problem arises in a number of standard and modern statistical learning models, including sparse principal component analysis, sparse Fisher discri…
Weighted iteration complexity of the sPADMM on the KKT residuals for convex composite optimization
Li Shen, Shaohua Pan
In this paper we establish an weighted iteration complexity on the KKT residuals yielded by the sPADMM (semi-proximal alternating direction method of multipl…
Linear convergence of the generalized PPA and several splitting methods for the composite inclusion problem
Li Shen, Shaohua Pan
For the inclusion problem involving two maximal monotone operators, under the metric subregularity of the composite operator, we derive the linear convergence of the generalized pr…
Inexact indefinite proximal ADMMs for 2-block separable convex programs and applications to 4-block DNNSDPs
Li Shen, Shaohua Pan
This paper is concerned with two-block separable convex minimization problems with linear constraints, for which it is either impossible or too expensive to obtain the exact soluti…
A corrected semi-proximal ADMM for multi-block convex optimization and its application to DNN-SDPs
Li Shen, Shaohua Pan
In this paper we propose a corrected semi-proximal ADMM (alternating direction method of multipliers) for the general -block convex optimization problems with linea…