CryptoDL: Deep Neural Networks over Encrypted Data

Abstract

Machine learning algorithms based on deep neural networks have achieved remarkable results and are being extensively used in different domains. However, the machine learning algorithms requires access to raw data which is often privacy sensitive. To address this issue, we develop new techniques to provide solutions for running deep neural networks over encrypted data. In this paper, we develop new techniques to adopt deep neural networks within the practical limitation of current homomorphic encryption schemes. More specifically, we focus on classification of the well-known convolutional neural networks (CNN). First, we design methods for approximation of the activation functions commonly used in CNNs (i.e. ReLU, Sigmoid, and Tanh) with low degree polynomials which is essential for efficient homomorphic encryption schemes. Then, we train convolutional neural networks with the approximation polynomials instead of original activation functions and analyze the performance of the models. Finally, we implement convolutional neural networks over encrypted data and measure performance of the models. Our experimental results validate the soundness of our approach with several convolutional neural networks with varying number of layers and structures. When applied to the MNIST optical character recognition tasks, our approach achieves 99.52\% accuracy which significantly outperforms the state-of-the-art solutions and is very close to the accuracy of the best non-private version, 99.77\%. Also, it can make close to 164000 predictions per hour. We also applied our approach to CIFAR-10, which is much more complex compared to MNIST, and were able to achieve 91.5\% accuracy with approximation polynomials used as activation functions. These results show that CryptoDL provides efficient, accurate and scalable privacy-preserving predictions.

Cited by in corpus (58)

• Federated Machine Learning: Concept and Applications
• Split learning for health: Distributed deep learning without sharing raw patient data
• Edge Intelligence: Architectures, Challenges, and Applications
• GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learning
• Privacy in Deep Learning: A Survey
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Security and Privacy Issues in Deep Learning
• BlindFL: Vertical Federated Machine Learning without Peeking into Your Data
• Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications
• Physical Side-Channel Attacks on Embedded Neural Networks: A Survey
• Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework
• AutoGAN-based Dimension Reduction for Privacy Preservation
• Secure Byzantine-Robust Machine Learning
• Multilinear Compressive Learning
• Input-Output History Feedback Controller for Encrypted Control with Leveled Fully Homomorphic Encryption
• Security for Machine Learning-based Systems: Attacks and Challenges during Training and Inference
• HEAX: An Architecture for Computing on Encrypted Data
• Enabling Homomorphically Encrypted Inference for Large DNN Models
• Privacy-Preserving Inference in Machine Learning Services Using Trusted Execution Environments
• A Scalable Approach for Privacy-Preserving Collaborative Machine Learning
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• SoK: Privacy Preserving Machine Learning using Functional Encryption: Opportunities and Challenges
• On Polynomial Approximations for Privacy-Preserving and Verifiable ReLU Networks
• SPEED: Secure, PrivatE, and Efficient Deep learning
• Towards the AlexNet Moment for Homomorphic Encryption: HCNN, theFirst Homomorphic CNN on Encrypted Data with GPUs
• nGraph-HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data
• CaRENets: Compact and Resource-Efficient CNN for Homomorphic Inference on Encrypted Medical Images
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges
• Is Private Learning Possible with Instance Encoding?
• Privacy-Preserving XGBoost Inference
• NN-EMD: Efficiently Training Neural Networks using Encrypted Multi-Sourced Datasets
• When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control
• CryptoNN: Training Neural Networks over Encrypted Data
• Privacy preserving Neural Network Inference on Encrypted Data with GPUs
• PANDA: Facilitating Usable AI Development
• SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation
• CECILIA: Comprehensive Secure Machine Learning Framework
• Convolutional Neural Networks on Randomized Data
• Privacy Preserving for Medical Image Analysis via Non-Linear Deformation Proxy
• Sisyphus: A Cautionary Tale of Using Low-Degree Polynomial Activations in Privacy-Preserving Deep Learning
• A brief history on Homomorphic learning: A privacy-focused approach to machine learning
• Stabilizing Inputs to Approximated Nonlinear Functions for Inference with Homomorphic Encryption in Deep Neural Networks
• VeriDL: Integrity Verification of Outsourced Deep Learning Services (Extended Version)
• Privacy-Preserving Medical Image Classification through Deep Learning and Matrix Decomposition
• Reinforcement Learning on Encrypted Data
• Enabling Inference Privacy with Adaptive Noise Injection
• Practical Two-party Privacy-preserving Neural Network Based on Secret Sharing
• MOFHEI: Model Optimizing Framework for Fast and Efficient Homomorphically Encrypted Neural Network Inference
• SIRNN: A Math Library for Secure RNN Inference
• A Novel Data Encryption Method Inspired by Adversarial Attacks
• Machine Learning in Precision Medicine to Preserve Privacy via Encryption
• Confidential Machine Learning on Untrusted Platforms: A Survey
• CryptoNite: Revealing the Pitfalls of End-to-End Private Inference at Scale
• Private Facial Diagnosis as an Edge Service for Parkinson's DBS Treatment Valuation
• Corella: A Private Multi Server Learning Approach based on Correlated Queries
• Multitask Identity-Aware Image Steganography via Minimax Optimization
• Obfuscation for Privacy-preserving Syntactic Parsing
• Accelerating 2PC-based ML with Limited Trusted Hardware