Local Differential Privacy for Deep Learning

Abstract

The internet of things (IoT) is transforming major industries including but not limited to healthcare, agriculture, finance, energy, and transportation. IoT platforms are continually improving with innovations such as the amalgamation of software-defined networks (SDN) and network function virtualization (NFV) in the edge-cloud interplay. Deep learning (DL) is becoming popular due to its remarkable accuracy when trained with a massive amount of data, such as generated by IoT. However, DL algorithms tend to leak privacy when trained on highly sensitive crowd-sourced data such as medical data. Existing privacy-preserving DL algorithms rely on the traditional server-centric approaches requiring high processing powers. We propose a new local differentially private (LDP) algorithm named LATENT that redesigns the training process. LATENT enables a data owner to add a randomization layer before data leave the data owners' devices and reach a potentially untrusted machine learning service. This feature is achieved by splitting the architecture of a convolutional neural network (CNN) into three layers: (1) convolutional module, (2) randomization module, and (3) fully connected module. Hence, the randomization module can operate as an NFV privacy preservation service in an SDN-controlled NFV, making LATENT more practical for IoT-driven cloud-based environments compared to existing approaches. The randomization module employs a newly proposed LDP protocol named utility enhancing randomization, which allows LATENT to maintain high utility compared to existing LDP protocols. Our experimental evaluation of LATENT on convolutional deep neural networks demonstrates excellent accuracy (e.g. 91%- 96%) with high model quality even under low privacy budgets (e.g. $\epsilon =0.5$).

References in corpus (9)

• XGBoost: A Scalable Tree Boosting System
• Deep Learning in Neural Networks: An Overview
• Deep Learning with Differential Privacy
• RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response
• Local, Private, Efficient Protocols for Succinct Histograms
• Efficient privacy preservation of big data for accurate data mining
• Efficient Data Perturbation for Privacy Preserving and Accurate Data Stream Mining
• An Efficient and Scalable Privacy Preserving Algorithm for Big Data and Data Streams
• MVG Mechanism: Differential Privacy under Matrix-Valued Query

Cited by in corpus (9)

• Edge Learning for B5G Networks with Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing
• A Comprehensive Survey on Local Differential Privacy Toward Data Statistics and Analysis
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Privacy-Preserving in Blockchain-based Federated Learning Systems
• Improving the utility of locally differentially private protocols for longitudinal and multidimensional frequency estimates
• Differentially Private Multivariate Time Series Forecasting of Aggregated Human Mobility With Deep Learning: Input or Gradient Perturbation?
• Quantum Local Differential Privacy and Quantum Statistical Query Model
• Tensorization of neural networks for improved privacy and interpretability
• PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction