339 citations · 392 across the 14 of their papers we have counts for
11 papers · 1 filter
On the Convergence and Stability of Distributed Sub-model Training
Yuyang Deng, Fuli Qiao, Mehrdad Mahdavi
As learning models continue to grow in size, enabling on-device local training of these models has emerged as a critical challenge in federated learning. A popular solution is sub-…
On the Generalization Ability of Unsupervised Pretraining
Yuyang Deng, Junyuan Hong, Jiayu Zhou +1
Recent advances in unsupervised learning have shown that unsupervised pre-training, followed by fine-tuning, can improve model generalization. However, a rigorous understanding of…
Distributed Personalized Empirical Risk Minimization
Yuyang Deng, Mohammad Mahdi Kamani, Pouria Mahdavinia +1
This paper advocates a new paradigm Personalized Empirical Risk Minimization (PERM) to facilitate learning from heterogeneous data sources without imposing stringent constraints on…
Understanding Deep Gradient Leakage via Inversion Influence Functions
Haobo Zhang, Junyuan Hong, Yuyang Deng +2
Deep Gradient Leakage (DGL) is a highly effective attack that recovers private training images from gradient vectors. This attack casts significant privacy challenges on distribute…
Mixture Weight Estimation and Model Prediction in Multi-source Multi-target Domain Adaptation
Yuyang Deng, Ilja Kuzborskij, Mehrdad Mahdavi
We consider the problem of learning a model from multiple heterogeneous sources with the goal of performing well on a new target distribution. The goal of learner is to mix these d…
On the Hardness of Robustness Transfer: A Perspective from Rademacher Complexity over Symmetric Difference Hypothesis Space
Yuyang Deng, Nidham Gazagnadou, Junyuan Hong +2
Recent studies demonstrated that the adversarially robust learning under attack is harder to generalize to different domains than standard domain adaptation. How to t…
Tight Analysis of Extra-gradient and Optimistic Gradient Methods For Nonconvex Minimax Problems
Pouria Mahdavinia, Yuyang Deng, Haochuan Li +1
Despite the established convergence theory of Optimistic Gradient Descent Ascent (OGDA) and Extragradient (EG) methods for the convex-concave minimax problems, little is known abou…
Local SGD Optimizes Overparameterized Neural Networks in Polynomial Time
Yuyang Deng, Mohammad Mahdi Kamani, Mehrdad Mahdavi
In this paper we prove that Local (S)GD (or FedAvg) can optimize deep neural networks with Rectified Linear Unit (ReLU) activation function in polynomial time. Despite the establis…
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency
Yuyang Deng, Mehrdad Mahdavi
Local SGD is a promising approach to overcome the communication overhead in distributed learning by reducing the synchronization frequency among worker nodes. Despite the recent th…
Distributionally Robust Federated Averaging
Yuyang Deng, Mohammad Mahdi Kamani, Mehrdad Mahdavi
In this paper, we study communication efficient distributed algorithms for distributionally robust federated learning via periodic averaging with adaptive sampling. In contrast to…
Adaptive Personalized Federated Learning
Yuyang Deng, Mohammad Mahdi Kamani, Mehrdad Mahdavi
Investigation of the degree of personalization in federated learning algorithms has shown that only maximizing the performance of the global model will confine the capacity of the…