Privacy-preserving Learning via Deep Net Pruning
arXiv:2003.01876
Abstract
This paper attempts to answer the question whether neural network pruning can be used as a tool to achieve differential privacy without losing much data utility. As a first step towards understanding the relationship between neural network pruning and differential privacy, this paper proves that pruning a given layer of the neural network is equivalent to adding a certain amount of differentially private noise to its hidden-layer activations. The paper also presents experimental results to show the practical implications of the theoretical finding and the key parameter values in a simple practical setting. These results show that neural network pruning can be a more effective alternative to adding differentially private noise for neural networks.
References in corpus (12)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Distilling the Knowledge in a Neural Network
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
- Deep Learning with Differential Privacy
- Federated Learning: Strategies for Improving Communication Efficiency
- To prune, or not to prune: exploring the efficacy of pruning for model compression
- The State of Sparsity in Deep Neural Networks
- Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
- Split learning for health: Distributed deep learning without sharing raw patient data
- Why are deep nets reversible: A simple theory, with implications for training
- Sample-Optimal Fourier Sampling in Any Constant Dimension -- Part I
- Sparsity Lower Bounds for Dimensionality Reducing Maps