2 citations · 3 across the 14 of their papers we have counts for
14 papers
Solution-space heterogeneity shapes federated learning dynamics across partial differential equations
Ping Luo, Jiahuan Wang, Ziqing Wen +2
Federated scientific machine learning enables institutions to train neural surrogates without centralizing local physical data, yet studies of partial differential equations (PDEs)…
Unveiling High-Probability Generalization in Decentralized SGD
Jiahuan Wang, Ping Luo, Ziqing Wen +2
Decentralized stochastic gradient descent (D-SGD) is an efficient method for large-scale distributed learning. Existing generalization studies mainly address expected results, achi…
MoLS: SNR-Guided Module-Wise Learning-Rate Calibration for Large Language Models
Ziqing Wen, Zhouyang Liu, Jiahuan Wang +5
Large language models (LLMs) contain functionally distinct modules with heterogeneous gradient statistics. Although Adam(W) adapts updates at the parameter level, selecting relativ…
Stability and Generalization for Decentralized Markov SGD
Jiahuan Wang, Ziqing Wen, Ping Luo +2
Stochastic gradient methods are central to large-scale learning, yet their generalization theory typically relies on independent sampling assumptions. In many practical application…
Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
Ping Luo, Jiahuan Wang, Ziqing Wen +2
Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its stability is fundamentally challenged by statistical heter…
FOAM: Blocked State Folding for Memory-Efficient LLM Training
Ziqing Wen, Jiahuan Wang, Ping Luo +2
Large language models (LLMs) have demonstrated remarkable performance due to their large parameter counts and extensive training data. However, their scale leads to significant mem…
GWT: Scalable Optimizer State Compression for Large Language Model Training
Ziqing Wen, Ping Luo, Jiahuan Wang +4
Training large language models (LLMs) requires substantial memory, a significant fraction of which is consumed by the moment states maintained by adaptive optimizers such as Adam.…
Federated Prediction-Powered Inference from Decentralized Data
Ping Luo, Xiaoge Deng, Ziqing Wen +2
In various domains, the increasing application of machine learning allows researchers to access inexpensive predictive data, which can be utilized as auxiliary data for statistical…
Score-based Generative Models with Adaptive Momentum
Ziqing Wen, Xiaoge Deng, Ping Luo +2
Score-based generative models have demonstrated significant practical success in data-generating tasks. The models establish a diffusion process that perturbs the ground truth data…
Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
Ping Luo, Xiaoge Deng, Ziqing Wen +2
Federated Learning (FL) is a distributed machine learning framework in communication network systems. However, the systems' Non-Independent and Identically Distributed (Non-IID) da…
Rethinking SIGN Training: Provable Nonconvex Acceleration without First- and Second-Order Gradient Lipschitz
Tao Sun, Congliang Chen, Peng Qiao +3
Sign-based stochastic methods have gained attention due to their ability to achieve robust performance despite using only the sign information for parameter updates. However, the c…
Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
Xiaoge Deng, Li Shen, Shengwei Li +3
Stochastic gradient descent (SGD) performed in an asynchronous manner plays a crucial role in training large-scale machine learning models. However, the generalization performance…
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient
Xiaoge Deng, Dongsheng Li, Tao Sun +1
Gradient-based optimization methods implemented on distributed computing architectures are increasingly used to tackle large-scale machine learning applications. A key bottleneck i…
Stability and Generalization of the Decentralized Stochastic Gradient Descent
Tao Sun, Dongsheng Li, Bao Wang
The stability and generalization of stochastic gradient-based methods provide valuable insights into understanding the algorithmic performance of machine learning models. As the ma…