Optimal Compression of Locally Differentially Private Mechanisms
arXiv:2111.00092
Abstract
Compressing the output of ε-locally differentially private (LDP) randomizers naively leads to suboptimal utility. In this work, we demonstrate the benefits of using schemes that jointly compress and privatize the data using shared randomness. In particular, we investigate a family of schemes based on Minimal Random Coding (Havasi et al., 2019) and prove that they offer optimal privacy-accuracy-communication tradeoffs. Our theoretical and empirical findings show that our approach can compress PrivUnit (Bhowmick et al., 2018) and Subset Selection (Ye et al., 2018), the best known LDP algorithms for mean and frequency estimation, to to the order of ε-bits of communication while preserving their privacy and accuracy guarantees.
References in corpus (13)
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
- TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
- Protection Against Reconstruction and Its Applications in Private Federated Learning
- ATOMO: Communication-efficient Learning via Atomic Sparsification
- Distributed Mean Estimation with Limited Communication
- Hadamard Response: Estimating Distributions Privately, Efficiently, and with Little Communication
- Practical Locally Private Heavy Hitters
- Breaking the Communication-Privacy-Accuracy Trilemma
- Communication Complexity in Locally Private Distribution Estimation and Heavy Hitters
- rTop-k: A Statistical Estimation Approach to Distributed SGD
- Locally Differentially Private Data Collection and Analysis
- Lossless Compression of Efficient Private Local Randomizers
- Breaking The Dimension Dependence in Sparse Distribution Estimation under Communication Constraints