Hermes Attack: Steal DNN Models with Lossless Inference Accuracy

Abstract

Deep Neural Networks (DNNs) models become one of the most valuable enterprise assets due to their critical roles in all aspects of applications. With the trend of privatization deployment of DNN models, the data leakage of the DNN models is becoming increasingly serious and widespread. All existing model-extraction attacks can only leak parts of targeted DNN models with low accuracy or high overhead. In this paper, we first identify a new attack surface -- unencrypted PCIe traffic, to leak DNN models. Based on this new attack surface, we propose a novel model-extraction attack, namely Hermes Attack, which is the first attack to fully steal the whole victim DNN model. The stolen DNN models have the same hyper-parameters, parameters, and semantically identical architecture as the original ones. It is challenging due to the closed-source CUDA runtime, driver, and GPU internals, as well as the undocumented data structures and the loss of some critical semantics in the PCIe traffic. Additionally, there are millions of PCIe packets with numerous noises and chaos orders. Our Hermes Attack addresses these issues by huge reverse engineering efforts and reliable semantic reconstruction, as well as skillful packet selection and order correction. We implement a prototype of the Hermes Attack, and evaluate two sequential DNN models (i.e., MINIST and VGG) and one consequential DNN model (i.e., ResNet) on three NVIDIA GPU platforms, i.e., NVIDIA Geforce GT 730, NVIDIA Geforce GTX 1080 Ti, and NVIDIA Geforce RTX 2080 Ti. The evaluation results indicate that our scheme is able to efficiently and completely reconstruct ALL of them with making inferences on any one image. Evaluated with Cifar10 test dataset that contains 10,000 images, the experiment results show that the stolen models have the same inference accuracy as the original ones (i.e., lossless inference accuracy).

arXiv commentThe paper will appear in Usenix Security Symposium 2021

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Houdini: Fooling Deep Structured Prediction Models
• Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach
• UPSET and ANGRI : Breaking High Performance Image Classifiers
• Stealing Neural Networks via Timing Side Channels
• Exploring the Space of Black-box Attacks on Deep Neural Networks
• Multi-column Deep Neural Networks for Image Classification
• Model Weight Theft With Just Noise Inputs: The Curious Case of the Petulant Attacker
• Open DNN Box by Power Side-Channel Attack

Cited by in corpus (2)

• Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective
• Delving into Data: Effectively Substitute Training for Black-box Attack