Differentially Private Bayesian Inference for Generalized Linear Models

Abstract

Generalized linear models (GLMs) such as logistic regression are among the most widely used arms in data analyst's repertoire and often used on sensitive datasets. A large body of prior works that investigate GLMs under differential privacy (DP) constraints provide only private point estimates of the regression coefficients, and are not able to quantify parameter uncertainty. In this work, with logistic and Poisson regression as running examples, we introduce a generic noise-aware DP Bayesian inference method for a GLM at hand, given a noisy sum of summary statistics. Quantifying uncertainty allows us to determine which of the regression coefficients are statistically significantly different from zero. We provide a previously unknown tight privacy analysis and experimentally demonstrate that the posteriors obtained from our model, while adhering to strong privacy guarantees, are close to the non-private posteriors.

References in corpus (13)

• Deep Learning with Differential Privacy
• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• Differentially Private Empirical Risk Minimization
• Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo
• Efficient, Differentially Private Point Estimators
• Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
• Differentially-Private "Draw and Discard" Machine Learning
• Differentially Private Bayesian Linear Regression
• Differentially Private Bayesian Inference for Exponential Families
• Locally Private Bayesian Inference for Count Models
• Estimating Smooth GLM in Non-interactive Local Differential Privacy Model with Public Unlabeled Data
• Differentially Private Markov Chain Monte Carlo
• PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference