8 papers
From Privacy to Generalization: Linear Max-Information Bounds for DP-SGD
Christoph H. Lampert, Hossein Zakerinia
Understanding the relationship between generalization and privacy remains a central challenge in modern machine learning theory, particularly for deep networks trained by variants…
Fast Rate Bounds for Multi-Task and Meta-Learning with Different Sample Sizes
Hossein Zakerinia, Christoph H. Lampert
We present new fast-rate PAC-Bayesian generalization bounds for multi-task and meta-learning in the unbalanced setting, i.e. when the tasks have training sets of different sizes, a…
Communication-Efficient Federated Learning With Data and Client Heterogeneity
Hossein Zakerinia, Shayan Talaei, Giorgi Nadiradze +1
Federated Learning (FL) enables large-scale distributed training of machine learning models, while still allowing individual nodes to maintain data locally. However, executing FL a…
Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions
Hossein Zakerinia, Jonathan Scott, Christoph H. Lampert
Personalized federated learning has emerged as a popular approach to training on devices holding statistically heterogeneous data, known as clients. However, most existing approach…
From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learning
Hossein Zakerinia, Dorsa Ghobadi, Christoph H. Lampert
Deep learning methods are known to generalize well from training to future data, even in an overparametrized regime, where they could easily overfit. One explanation for this pheno…
PeFLL: Personalized Federated Learning by Learning to Learn
Jonathan Scott, Hossein Zakerinia, Christoph H. Lampert
We present PeFLL, a new personalized federated learning algorithm that improves over the state-of-the-art in three aspects: 1) it produces more accurate models, especially in the l…