NeuralDP Differentially private neural networks by design

Abstract

The application of differential privacy to the training of deep neural networks holds the promise of allowing large-scale (decentralized) use of sensitive data while providing rigorous privacy guarantees to the individual. The predominant approach to differentially private training of neural networks is DP-SGD, which relies on norm-based gradient clipping as a method for bounding sensitivity, followed by the addition of appropriately calibrated Gaussian noise. In this work we propose NeuralDP, a technique for privatising activations of some layer within a neural network, which by the post-processing properties of differential privacy yields a differentially private network. We experimentally demonstrate on two datasets (MNIST and Pediatric Pneumonia Dataset (PPD)) that our method offers substantially improved privacy-utility trade-offs compared to DP-SGD.

arXiv commentPaper withdrawn. The paper contains a factual error

References in corpus (13)

• Spectral Normalization for Generative Adversarial Networks
• Improved Training of Wasserstein GANs
• Renyi Differential Privacy
• Robustness May Be at Odds with Accuracy
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Parseval Networks: Improving Robustness to Adversarial Examples
• Gaussian Differential Privacy
• Subsampled Rényi Differential Privacy and Analytical Moments Accountant
• Noisy Activation Functions
• Sorting out Lipschitz function approximation
• Understanding Gradient Clipping in Private SGD: A Geometric Perspective
• Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Results
• Efficient Lipschitz Extensions for High-Dimensional Graph Statistics and Node Private Degree Distributions