21 citations · 78 across the 16 of their papers we have counts for
18 papers
Polynomial Acceleration of Alternating Projections: Chebyshev Acceleration, Convergence Rates, and Retractions
Dengyu Zheng, Shixiang Chen
Alternating projections (AP) near a clean intersection of smooth manifolds can converge slowly when the Friedrichs angle is small. We develop the \emph{Fixed-Point Manifold Acceler…
From Manifold Identification to Newton Acceleration on Intersections: Sparse Stiefel Optimization
Shixiang Chen, Wen Huang
We study a Newton acceleration for sparse composite optimization on the Stiefel manifold. The main difficulty is geometric: the active manifold identified by the nonsmooth regulari…
Retractions by Alternating Projections
Shixiang Chen, Yixiao He, Wen Huang
Alternating projections and their variants are classical tools for computing points in intersections of sets. Existing analyses for smooth manifolds mainly focus on local convergen…
Distributed Stochastic Proximal Algorithm on Riemannian Submanifolds for Weakly-convex Functions
Jishu Zhao, Xi Wang, Jinlong Lei +1
This paper aims to investigate the distributed stochastic optimization problems on compact embedded submanifolds (in the Euclidean space) where the local cost functions are weakly-…
Retraction-Free Decentralized Non-convex Optimization with Orthogonal Constraints
Youbang Sun, Shixiang Chen, Alfredo Garcia +1
In this paper, we investigate decentralized non-convex optimization with orthogonal constraints. Conventional algorithms for this setting require either manifold retractions or oth…
FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data
Hao Sun, Li Shen, Shixiang Chen +4
Federated learning is an emerging distributed machine learning method, enables a large number of clients to train a model without exchanging their local data. The time cost of comm…
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape
Yan Sun, Li Shen, Shixiang Chen +2
In federated learning (FL), a cluster of local clients are chaired under the coordination of the global server and cooperatively train one model with privacy protection. Due to the…
Decentralized Weakly Convex Optimization Over the Stiefel Manifold
Jinxin Wang, Jiang Hu, Shixiang Chen +2
We focus on a class of non-smooth optimization problems over the Stiefel manifold in the decentralized setting, where a connected network of agents cooperatively minimize a fin…
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Hao Sun, Li Shen, Qihuang Zhong +6
Sharpness aware minimization (SAM) optimizer has been extensively explored as it can generalize better for training deep neural networks via introducing extra perturbation steps to…
OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System
Chao Xue, Wei Liu, Shuai Xie +25
Automated machine learning (AutoML) seeks to build ML models with minimal human effort. While considerable research has been conducted in the area of AutoML in general, aiming to t…
Penalized Proximal Policy Optimization for Safe Reinforcement Learning
Linrui Zhang, Li Shen, Long Yang +4
Safe reinforcement learning aims to learn the optimal policy while satisfying safety constraints, which is essential in real-world applications. However, current algorithms still s…
Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?
Yibo Yang, Shixiang Chen, Xiangtai Li +3
Modern deep neural networks for classification usually jointly learn a backbone for representation and a linear classifier to output the logit of each class. A recent study has sho…
A Manifold Proximal Linear Method for Sparse Spectral Clustering with Application to Single-Cell RNA Sequencing Data Analysis
Zhongruo Wang, Bingyuan Liu, Shixiang Chen +3
Spectral clustering is one of the fundamental unsupervised learning methods widely used in data analysis. Sparse spectral clustering (SSC) imposes sparsity to the spectral clusteri…
Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learning
Shixiang Chen, Zengde Deng, Shiqian Ma +1
We consider the problem of maximizing the norm of a linear map over the sphere, which arises in various machine learning applications such as orthogonal dictionary learnin…
Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods
Xiao Li, Shixiang Chen, Zengde Deng +3
We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which the objective function is weakly convex in the ambient Euclidean space. Such problems are…
An Alternating Manifold Proximal Gradient Method for Sparse PCA and Sparse CCA
Shixiang Chen, Shiqian Ma, Lingzhou Xue +1
Sparse principal component analysis (PCA) and sparse canonical correlation analysis (CCA) are two essential techniques from high-dimensional statistics and machine learning for ana…
Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
Shixiang Chen, Shiqian Ma, Anthony Man-Cho So +1
We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving th…
Geometric descent method for convex composite minimization
Shixiang Chen, Shiqian Ma, Wei Liu
In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our propo…