Privacy in Deep Learning: A Survey

Abstract

The ever-growing advances of deep learning in many areas including vision, recommendation systems, natural language processing, etc., have led to the adoption of Deep Neural Networks (DNNs) in production systems. The availability of large datasets and high computational power are the main contributors to these advances. The datasets are usually crowdsourced and may contain sensitive information. This poses serious privacy concerns as this data can be misused or leaked through various vulnerabilities. Even if the cloud provider and the communication link is trusted, there are still threats of inference attacks where an attacker could speculate properties of the data used for training, or find the underlying model architecture and parameters. In this survey, we review the privacy concerns brought by deep learning, and the mitigating techniques introduced to tackle these issues. We also show that there is a gap in the literature regarding test-time inference privacy, and propose possible future research directions.

References in corpus (40)

• Deep Learning with Differential Privacy
• Geo-Indistinguishability: Differential Privacy for Location-Based Systems
• Learning Differentially Private Recurrent Language Models
• Poisoning Attacks against Support Vector Machines
• Stealing Machine Learning Models via Prediction APIs
• Differentially Private Federated Learning: A Client Level Perspective
• Adversarial Attacks and Defences: A Survey
• A Closer Look at Memorization in Deep Networks
• Scalable Private Learning with PATE
• Differentially Private Generative Adversarial Network
• CryptoDL: Deep Neural Networks over Encrypted Data
• Mobile Sensor Data Anonymization
• Protection Against Reconstruction and Its Applications in Private Federated Learning
• Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
• DarkneTZ: Towards Model Privacy at the Edge using Trusted Execution Environments
• Split learning for health: Distributed deep learning without sharing raw patient data
• XONN: XNOR-based Oblivious Deep Neural Network Inference
• Gaussian Differential Privacy
• Analyzing Information Leakage of Updates to Natural Language Models
• Towards Demystifying Membership Inference Attacks
• Auditing Differentially Private Machine Learning: How Private is Private SGD?
• Privacy-preserving Machine Learning through Data Obfuscation
• Scalable and Differentially Private Distributed Aggregation in the Shuffled Model
• Towards Measuring Membership Privacy
• TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service
• Learning Privately from Multiparty Data
• Encrypted statistical machine learning: new privacy preserving methods
• Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures
• QUOTIENT: Two-Party Secure Neural Network Training and Prediction
• Privacy-Preserving Distributed Deep Learning for Clinical Data
• Privacy-Preserving Inference in Machine Learning Services Using Trusted Execution Environments
• Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment
• Deep Learning with Gaussian Differential Privacy
• Secure Face Matching Using Fully Homomorphic Encryption
• Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study
• Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
• A Non-Parametric Test to Detect Data-Copying in Generative Models
• DarKnight: A Data Privacy Scheme for Training and Inference of Deep Neural Networks
• Membership Inference Attacks against Machine Learning Models
• Obfuscation via Information Density Estimation

Cited by in corpus (19)

• FedML: A Research Library and Benchmark for Federated Machine Learning
• Interference Suppression Using Deep Learning: Current Approaches and Open Challenges
• Unsupervised Deep Learning for IoT Time Series
• Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey
• Socially Responsible AI Algorithms: Issues, Purposes, and Challenges
• Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs
• Collaborative Intelligence: Challenges and Opportunities
• Hierarchical Training of Deep Neural Networks Using Early Exiting
• Membership Inference on Word Embedding and Beyond
• Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective
• Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy
• Formal Methods for Autonomous Systems
• Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
• DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?
• A Lightweight Privacy-Preserving Scheme Using Label-based Pixel Block Mixing for Image Classification in Deep Learning
• Federated Learning for Open Banking
• On the Privacy Risks of Deploying Recurrent Neural Networks in Machine Learning Models
• When Differential Privacy Meets Interpretability: A Case Study
• Privacy and Integrity Preserving Training Using Trusted Hardware