Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures

Abstract

Deep Neural Networks (DNNs) are fast becoming ubiquitous for their ability to attain good accuracy in various machine learning tasks. A DNN's architecture (i.e., its hyper-parameters) broadly determines the DNN's accuracy and performance, and is often confidential. Attacking a DNN in the cloud to obtain its architecture can potentially provide major commercial value. Further, attaining a DNN's architecture facilitates other, existing DNN attacks. This paper presents Cache Telepathy: a fast and accurate mechanism to steal a DNN's architecture using the cache side channel. Our attack is based on the insight that DNN inference relies heavily on tiled GEMM (Generalized Matrix Multiply), and that DNN architecture parameters determine the number of GEMM calls and the dimensions of the matrices used in the GEMM functions. Such information can be leaked through the cache side channel. This paper uses Prime+Probe and Flush+Reload to attack VGG and ResNet DNNs running OpenBLAS and Intel MKL libraries. Our attack is effective in helping obtain the architectures by very substantially reducing the search space of target DNN architectures. For example, for VGG using OpenBLAS, it reduces the search space from more than $10^{35}$ architectures to just 16.

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Deep Residual Learning for Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Stealing Machine Learning Models via Prediction APIs
• Understanding Membership Inferences on Well-Generalized Learning Models
• Stealing Hyperparameters in Machine Learning

Cited by in corpus (13)

• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Privacy in Deep Learning: A Survey
• Stealing Neural Networks via Timing Side Channels
• Evaluating Differentially Private Machine Learning in Practice
• Security Analysis of Deep Neural Networks Operating in the Presence of Cache Side-Channel Attacks
• Physical Side-Channel Attacks on Embedded Neural Networks: A Survey
• A framework for the extraction of Deep Neural Networks by leveraging public data
• IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary
• SPEED: Secure, PrivatE, and Efficient Deep learning
• Stealing Deep Reinforcement Learning Models for Fun and Profit
• SEALing Neural Network Models in Secure Deep Learning Accelerators
• SpecuSym: Speculative Symbolic Execution for Cache Timing Leak Detection
• Ten AI Stepping Stones for Cybersecurity