Differentially Private Markov Chain Monte Carlo

Abstract

Recent developments in differentially private (DP) machine learning and DP Bayesian learning have enabled learning under strong privacy guarantees for the training data subjects. In this paper, we further extend the applicability of DP Bayesian learning by presenting the first general DP Markov chain Monte Carlo (MCMC) algorithm whose privacy-guarantees are not subject to unrealistic assumptions on Markov chain convergence and that is applicable to posterior inference in arbitrary models. Our algorithm is based on a decomposition of the Barker acceptance test that allows evaluating the Rényi DP privacy cost of the accept-reject choice. We further show how to improve the DP guarantee through data subsampling and approximate acceptance tests.

arXiv comment22 pages, 12 figures

Cited by in corpus (8)

• Being Bayesian in the 2020s: opportunities and challenges in the practice of modern applied Bayesian statistics
• Differentially Private Bayesian Inference for Generalized Linear Models
• Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability
• Statistic Selection and MCMC for Differentially Private Bayesian Estimation
• Upper Bounds on the Generalization Error of Private Algorithms for Discrete Data
• Differentially Private Hamiltonian Monte Carlo
• Locally Differentially Private Bayesian Inference
• Gaussian Processes with Differential Privacy