Distributed Differentially Private Computation of Functions with Correlated Noise

Abstract

Many applications of machine learning, such as human health research, involve processing private or sensitive information. Privacy concerns may impose significant hurdles to collaboration in scenarios where there are multiple sites holding data and the goal is to estimate properties jointly across all datasets. Differentially private decentralized algorithms can provide strong privacy guarantees. However, the accuracy of the joint estimates may be poor when the datasets at each site are small. This paper proposes a new framework, Correlation Assisted Private Estimation (CAPE), for designing privacy-preserving decentralized algorithms with better accuracy guarantees in an honest-but-curious model. CAPE can be used in conjunction with the functional mechanism for statistical and machine learning optimization problems. A tighter characterization of the functional mechanism is provided that allows CAPE to achieve the same performance as a centralized algorithm in the decentralized setting using all datasets. Empirical results on regression and neural network problems for both synthetic and real datasets show that differentially private methods can be competitive with non-private algorithms in many scenarios of interest.

arXiv commentThe manuscript is partially subsumed by arXiv:1910.12913

References in corpus (11)

• Deep Learning with Differential Privacy
• Differentially Private Distributed Constrained Optimization
• Differential Privacy and Machine Learning: a Survey and Review
• Practical Secure Aggregation for Federated Learning on User-Held Data
• Distributed Mean Estimation with Limited Communication
• CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning
• Distributed Differentially-Private Algorithms for Matrix and Tensor Factorization
• Differentially Private Bayesian Learning on Distributed Data
• Accuracy First: Selecting a Differential Privacy Level for Accuracy-Constrained ERM
• Optimal Algorithms for Distributed Optimization
• Private Learning on Networks

Cited by in corpus (2)

• Privacy Preserving Set-Based Estimation Using Partially Homomorphic Encryption
• Improved Differentially Private Decentralized Source Separation for fMRI Data