4 papers · 1 filter
Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transfers
Fan Yang, Hongyang R. Zhang, Sen Wu +2
The problem of learning one task using samples from another task is central to transfer learning. In this paper, we focus on answering the following question: when does combining t…
Minimax Estimation for Personalized Federated Learning: An Alternative between FedAvg and Local Training?
Shuxiao Chen, Qinqing Zheng, Qi Long +1
A widely recognized difficulty in federated learning arises from the statistical heterogeneity among clients: local datasets often originate from distinct yet not entirely unrelate…
HiGrad: Uncertainty Quantification for Online Learning and Stochastic Approximation
Weijie J. Su, Yuancheng Zhu
Stochastic gradient descent (SGD) is an immensely popular approach for online learning in settings where data arrives in a stream or data sizes are very large. However, despite an…
Robust Learning Rate Selection for Stochastic Optimization via Splitting Diagnostic
Matteo Sordello, Niccolò Dalmasso, Hangfeng He +1
This paper proposes SplitSGD, a new dynamic learning rate schedule for stochastic optimization. This method decreases the learning rate for better adaptation to the local geometry…