Element Level Differential Privacy: The Right Granularity of Privacy

Abstract

Differential Privacy (DP) provides strong guarantees on the risk of compromising a user's data in statistical learning applications, though these strong protections make learning challenging and may be too stringent for some use cases. To address this, we propose element level differential privacy, which extends differential privacy to provide protection against leaking information about any particular "element" a user has, allowing better utility and more robust results than classical DP. By carefully choosing these "elements," it is possible to provide privacy protections at a desired granularity. We provide definitions, associated privacy guarantees, and analysis to identify the tradeoffs with the new definition; we also develop several private estimation and learning methodologies, providing careful examples for item frequency and M-estimation (empirical risk minimization) with concomitant privacy and utility analysis. We complement our theoretical and methodological advances with several real-world applications, estimating histograms and fitting several large-scale prediction models, including deep networks.

arXiv comment34 pages, 5 figures

References in corpus (8)

• Deep Learning with Differential Privacy
• Renyi Differential Privacy
• Geo-Indistinguishability: Differential Privacy for Location-Based Systems
• Learning Differentially Private Recurrent Language Models
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Gaussian Differential Privacy
• Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity
• Privacy and Statistical Risk: Formalisms and Minimax Bounds

Cited by in corpus (3)

• Layer-wise Characterization of Latent Information Leakage in Federated Learning
• Context-Aware Local Differential Privacy
• HydraGAN A Multi-head, Multi-objective Approach to Synthetic Data Generation