4 papers
FedSubMuon: Communication-Efficient Federated LLM Fine-Tuning via Structured Subspace Muon
Shaolong Chen, Youming Tao, Shuzhen Chen +3
Federated fine-tuning adapts large language models (LLMs) to decentralized client data, but its scalability in cross-device training is often limited by the high communication cost…
A successive difference-of-convex method for a class of two-stage nonconvex nonsmooth stochastic conic program via SVI
Chao Zhang, Di Wang
We consider a class of two-stage nonconvex nonsmooth stochastic conic program, where the objective functions in both stages can contain nonsmooth terms that are functions with easi…
Finding Differentially Private Second Order Stationary Points in Stochastic Minimax Optimization
Difei Xu, Youming Tao, Meng Ding +2
We provide the first study of the problem of finding differentially private (DP) second-order stationary points (SOSP) in stochastic (non-convex) minimax optimization. Existing lit…
Second-Order Convergence in Private Stochastic Non-Convex Optimization
Youming Tao, Zuyuan Zhang, Dongxiao Yu +3
We investigate the problem of finding second-order stationary points (SOSP) in differentially private (DP) stochastic non-convex optimization. Existing methods suffer from two key…