Scalable Differential Privacy with Certified Robustness in Adversarial Learning

Abstract

In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples. By leveraging the sequential composition theory in DP, we randomize both input and latent spaces to strengthen our certified robustness bounds. To address the trade-off among model utility, privacy loss, and robustness, we design an original adversarial objective function, based on the post-processing property in DP, to tighten the sensitivity of our model. A new stochastic batch training is proposed to apply our mechanism on large DNNs and datasets, by bypassing the vanilla iterative batch-by-batch training in DP DNNs. An end-to-end theoretical analysis and evaluations show that our mechanism notably improves the robustness and scalability of DP DNNs.

arXiv commentTo appear at ICML'2020 (29 pages)

Cited by in corpus (11)

• From Distributed Machine Learning to Federated Learning: A Survey
• When Machine Learning Meets Privacy: A Survey and Outlook
• Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
• Large Scale Private Learning via Low-rank Reparametrization
• Deep Learning with Label Differential Privacy
• Robustness Threats of Differential Privacy
• Privacy-preserving Collaborative Learning with Automatic Transformation Search
• Robust and Private Learning of Halfspaces
• Differentially Private Adversarial Robustness Through Randomized Perturbations
• Output Perturbation for Differentially Private Convex Optimization: Faster and More General
• CAPE: Context-Aware Private Embeddings for Private Language Learning