Rényi Differential Privacy of the Sampled Gaussian Mechanism

Abstract

The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadratically, rather than linearly, with the sampling rate. Characterizing the precise privacy properties of SGM motivated development of several relaxations of the notion of differential privacy. This work unifies and fills in gaps in published results on SGM. We describe a numerically stable procedure for precise computation of SGM's Rényi Differential Privacy and prove a nearly tight (within a small constant factor) closed-form bound.

arXiv comment14 pages

References in corpus (3)

• Deep Learning with Differential Privacy
• Rényi Divergence and Kullback-Leibler Divergence
• Renyi Differential Privacy

Cited by in corpus (38)

• How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
• Differentially Private Synthetic Medical Data Generation using Convolutional GANs
• Auditing Differentially Private Machine Learning: How Private is Private SGD?
• Differentially Private Learning Needs Better Features (or Much More Data)
• DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation
• The Skellam Mechanism for Differentially Private Federated Learning
• Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
• The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation
• Opacus: User-Friendly Differential Privacy Library in PyTorch
• Improving Deep Learning with Differential Privacy using Gradient Encoding and Denoising
• Evaluating Privacy-Preserving Machine Learning in Critical Infrastructures: A Case Study on Time-Series Classification
• Gradient Sparsification for Efficient Wireless Federated Learning with Differential Privacy
• Differentially Private Federated Learning for Cancer Prediction
• Large Scale Private Learning via Low-rank Reparametrization
• Computing Tight Differential Privacy Guarantees Using FFT
• Efficient Privacy-Preserving Stochastic Nonconvex Optimization
• Differentially Private Federated Learning with Laplacian Smoothing
• Wide Network Learning with Differential Privacy
• Kamino: Constraint-Aware Differentially Private Data Synthesis
• Privacy-Preserving Models for Legal Natural Language Processing
• On the Importance of Difficulty Calibration in Membership Inference Attacks
• Hyperparameter Tuning with Renyi Differential Privacy
• Renyi Differential Privacy of the Subsampled Shuffle Model in Distributed Learning
• Federated Learning in Adversarial Settings
• Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability
• The Trade-Offs of Private Prediction
• Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT
• Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFT
• DP-REC: Private & Communication-Efficient Federated Learning
• Enhancing the Antidote: Improved Pointwise Certifications against Poisoning Attacks
• Adaptive Differentially Private Empirical Risk Minimization
• Constrained Differentially Private Federated Learning for Low-bandwidth Devices
• High-Dimensional Private Empirical Risk Minimization by Greedy Coordinate Descent
• Differentially Private Generation of Small Images
• Private Knowledge Transfer via Model Distillation with Generative Adversarial Networks
• Privacy Amplification Via Bernoulli Sampling
• A Privacy-Preserving and Trustable Multi-agent Learning Framework
• Controlled privacy leakage propagation throughout overlapping grouped learning