Photonic Differential Privacy with Direct Feedback Alignment
arXiv:2106.03645
Abstract
Optical Processing Units (OPUs) -- low-power photonic chips dedicated to large scale random projections -- have been used in previous work to train deep neural networks using Direct Feedback Alignment (DFA), an effective alternative to backpropagation. Here, we demonstrate how to leverage the intrinsic noise of optical random projections to build a differentially private DFA mechanism, making OPUs a solution of choice to provide a private-by-design training. We provide a theoretical analysis of our adaptive privacy mechanism, carefully measuring how the noise of optical random projections propagates in the process and gives rise to provable Differential Privacy. Finally, we conduct experiments demonstrating the ability of our learning procedure to achieve solid end-task performance.
References in corpus (9)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- 11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks
- Renyi Differential Privacy
- Scalable Private Learning with PATE
- Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising
- Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks
- Kernel computations from large-scale random features obtained by Optical Processing Units
- Removing Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracy
- Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures