The Cost of Privacy in Generalized Linear Models: Algorithms and Minimax Lower Bounds
arXiv:2011.03900
Abstract
We propose differentially private algorithms for parameter estimation in both low-dimensional and high-dimensional sparse generalized linear models (GLMs) by constructing private versions of projected gradient descent. We show that the proposed algorithms are nearly rate-optimal by characterizing their statistical performance and establishing privacy-constrained minimax lower bounds for GLMs. The lower bounds are obtained via a novel technique, which is based on Stein's Lemma and generalizes the tracing attack technique for privacy-constrained lower bounds. This lower bound argument can be of independent interest as it is applicable to general parametric models. Simulated and real data experiments are conducted to demonstrate the numerical performance of our algorithms.
arXiv comment56 pages, 6 figures
References in corpus (10)
- Deep Learning with Differential Privacy
- Renyi Differential Privacy
- A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers
- Gaussian Differential Privacy
- On Iterative Hard Thresholding Methods for High-dimensional M-Estimation
- Privacy and Statistical Risk: Formalisms and Minimax Bounds
- The Cost of Privacy: Rates of Convergence for Parameter Estimation with Differential Privacy
- Differentially Private False Discovery Rate Control
- Quantifying the Privacy Risks of Learning High-Dimensional Graphical Models
- Privacy-Preserving Causal Inference via Inverse Probability Weighting