Controlled privacy leakage propagation throughout overlapping grouped learning

Abstract

Federated Learning (FL) is the standard protocol for collaborative learning. In FL, multiple workers jointly train a shared model. They exchange model updates calculated on their data, while keeping the raw data itself local. Since workers naturally form groups based on common interests and privacy policies, we are motivated to extend standard FL to reflect a setting with multiple, potentially overlapping groups. In this setup where workers can belong and contribute to more than one group at a time, complexities arise in understanding privacy leakage and in adhering to privacy policies. To address the challenges, we propose differential private overlapping grouped learning (DPOGL), a novel method to implement privacy guarantees within overlapping groups. Under the honest-but-curious threat model, we derive novel privacy guarantees between arbitrary pairs of workers. These privacy guarantees describe and quantify two key effects of privacy leakage in DP-OGL: propagation delay, i.e., the fact that information from one group will leak to other groups only with temporal offset through the common workers and information degradation, i.e., the fact that noise addition over model updates limits information leakage between workers. Our experiments show that applying DP-OGL enhances utility while maintaining strong privacy compared to standard FL setups.

This paper was presented in part at the 2024 IEEE International Symposium on Information Theory (ISIT), Athens, Greece, 2024, pp. 386-391, doi: 10.1109/ISIT57864.2024.10619521. The paper was published in the IEEE Journal on Selected Areas in Information Theory (JSAIT)

References in corpus (11)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Renyi Differential Privacy
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• Adaptive Personalized Federated Learning
• Three Approaches for Personalization with Applications to Federated Learning
• FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy
• Rényi Differential Privacy of the Sampled Gaussian Mechanism
• Unleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services
• Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent
• Have it your way: Individualized Privacy Assignment for DP-SGD