Differentially Private Bayesian Learning on Distributed Data

Abstract

Many applications of machine learning, for example in health care, would benefit from methods that can guarantee privacy of data subjects. Differential privacy (DP) has become established as a standard for protecting learning results. The standard DP algorithms require a single trusted party to have access to the entire data, which is a clear weakness. We consider DP Bayesian learning in a distributed setting, where each party only holds a single sample or a few samples of the data. We propose a learning strategy based on a secure multi-party sum function for aggregating summaries from data holders and the Gaussian mechanism for DP. Our method builds on an asymptotically optimal and practically efficient DP Bayesian inference with rapidly diminishing extra cost.

arXiv comment13 pages, 7 figures. Modified text, changed algorithm used, included tests on additional dataset, fixed several errors, added proof of asymptotic efficiency to supplement

Cited by in corpus (14)

• Differentially private cross-silo federated learning
• Reviewing and Improving the Gaussian Mechanism for Differential Privacy
• A Theoretical Perspective on Differentially Private Federated Multi-task Learning
• Privacy-preserving data sharing via probabilistic modelling
• Parametric Bootstrap for Differentially Private Confidence Intervals
• Efficient Privacy-Preserving Stochastic Nonconvex Optimization
• Differentially Private Gaussian Processes
• Distributed Differentially Private Computation of Functions with Correlated Noise
• Improved Differentially Private Decentralized Source Separation for fMRI Data
• Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability
• Towards practical differentially private causal graph discovery
• Local Differential Privacy in Decentralized Optimization
• Privacy-Preserving Multiparty Learning For Logistic Regression
• Secure and Differentially Private Bayesian Learning on Distributed Data