Optimal Accounting of Differential Privacy via Characteristic Function

Abstract

Characterizing the privacy degradation over compositions, i.e., privacy accounting, is a fundamental topic in differential privacy (DP) with many applications to differentially private machine learning and federated learning. We propose a unification of recent advances (Renyi DP, privacy profiles, $f$-DP and the PLD formalism) via the \emph{characteristic function} ($\varphi$-function) of a certain \emph{dominating} privacy loss random variable. We show that our approach allows \emph{natural} adaptive composition like Renyi DP, provides \emph{exactly tight} privacy accounting like PLD, and can be (often \emph{losslessly}) converted to privacy profile and $f$-DP, thus providing $(\epsilon ,\delta )$-DP guarantees and interpretable tradeoff functions. Algorithmically, we propose an \emph{analytical Fourier accountant} that represents the \emph{complex} logarithm of $\varphi$-functions symbolically and uses Gaussian quadrature for numerical computation. On several popular DP mechanisms and their subsampled counterparts, we demonstrate the flexibility and tightness of our approach in theory and experiments.

References in corpus (9)

• Deep Learning with Differential Privacy
• Rényi Divergence and Kullback-Leibler Divergence
• Renyi Differential Privacy
• Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences
• Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo
• Subsampled Rényi Differential Privacy and Analytical Moments Accountant
• The Discrete Gaussian for Differential Privacy
• Hypothesis Testing Interpretations and Renyi Differential Privacy
• Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFT

Cited by in corpus (3)

• An In-Depth Examination of Requirements for Disclosure Risk Assessment
• Dynamic Differential-Privacy Preserving SGD
• Complex-valued Federated Learning with Differential Privacy and MRI Applications