Federated Heavy Hitters Discovery with Differential Privacy

Abstract

The discovery of heavy hitters (most frequent items) in user-generated data streams drives improvements in the app and web ecosystems, but can incur substantial privacy risks if not done with care. To address these risks, we propose a distributed and privacy-preserving algorithm for discovering the heavy hitters in a population of user-generated data streams. We leverage the sampling and thresholding properties of our distributed algorithm to prove that it is inherently differentially private, without requiring additional noise. We also examine the trade-off between privacy and utility, and show that our algorithm provides excellent utility while also achieving strong privacy guarantees. A significant advantage of this approach is that it eliminates the need to centralize raw data while also avoiding the significant loss in utility incurred by local differential privacy. We validate our findings both theoretically, using worst-case analyses, and practically, using a Twitter dataset with 1.6M tweets and over 650k users. Finally, we carefully compare our approach to Apple's local differential privacy method for discovering heavy hitters.

References in corpus (9)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Learning Differentially Private Recurrent Language Models
• Differentially Private Federated Learning: A Client Level Perspective
• Extremal Mechanisms for Local Differential Privacy
• Discrete Distribution Estimation under Local Privacy
• Practical Secure Aggregation for Federated Learning on User-Held Data
• Hadamard Response: Estimating Distributions Privately, Efficiently, and with Little Communication
• PriPeARL: A Framework for Privacy-Preserving Analytics and Reporting at LinkedIn

Cited by in corpus (6)

• FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
• Personalized Federated Learning with Moreau Envelopes
• Differential Privacy Has Disparate Impact on Model Accuracy
• Privacy for Free: Communication-Efficient Learning with Differential Privacy Using Sketches
• Personalized Federated Learning using Hypernetworks
• A Federated Learning Approach for Mobile Packet Classification