Personalized Graph Federated Learning with Differential Privacy

Abstract

This paper presents a personalized graph federated learning (PGFL) framework in which distributedly connected servers and their respective edge devices collaboratively learn device or cluster-specific models while maintaining the privacy of every individual device. The proposed approach exploits similarities among different models to provide a more relevant experience for each device, even in situations with diverse data distributions and disproportionate datasets. Furthermore, to ensure a secure and efficient approach to collaborative personalized learning, we study a variant of the PGFL implementation that utilizes differential privacy, specifically zero-concentrated differential privacy, where a noise sequence perturbs model exchanges. Our mathematical analysis shows that the proposed privacy-preserving PGFL algorithm converges to the optimal cluster-specific solution for each cluster in linear time. It also shows that exploiting similarities among clusters leads to an alternative output whose distance to the original solution is bounded, and that this bound can be adjusted by modifying the algorithm's hyperparameters. Further, our analysis shows that the algorithm ensures local differential privacy for all clients in terms of zero-concentrated differential privacy. Finally, the performance of the proposed PGFL algorithm is examined by performing numerical experiments in the context of regression and classification using synthetic data and the MNIST dataset.

References in corpus (11)

• Federated Learning: Challenges, Methods, and Future Directions
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Towards Personalized Federated Learning
• MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification
• On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
• Personalized Federated Learning: A Meta-Learning Approach
• Adaptive Personalized Federated Learning
• Multitask Diffusion Adaptation over Networks
• Electrical Load Forecasting Using Edge Computing and Federated Learning
• Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning
• Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning