Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data

Abstract

In the past few years, Convolutional Neural Networks (CNNs) have been achieving state-of-the-art performance on a variety of problems. Many companies employ resources and money to generate these models and provide them as an API, therefore it is in their best interest to protect them, i.e., to avoid that someone else copies them. Recent studies revealed that state-of-the-art CNNs are vulnerable to adversarial examples attacks, and this weakness indicates that CNNs do not need to operate in the problem domain (PD). Therefore, we hypothesize that they also do not need to be trained with examples of the PD in order to operate in it. Given these facts, in this paper, we investigate if a target black-box CNN can be copied by persuading it to confess its knowledge through random non-labeled data. The copy is two-fold: i) the target network is queried with random data and its predictions are used to create a fake dataset with the knowledge of the network; and ii) a copycat network is trained with the fake dataset and should be able to achieve similar performance as the target network. This hypothesis was evaluated locally in three problems (facial expression, object, and crosswalk classification) and against a cloud-based API. In the copy attacks, images from both non-problem domain and PD were used. All copycat networks achieved at least 93.7% of the performance of the original models with non-problem domain data, and at least 98.6% using additional data from the PD. Additionally, the copycat CNN successfully copied at least 97.3% of the performance of the Microsoft Azure Emotion API. Our results show that it is possible to create a copycat CNN by simply querying a target network as black-box with random non-labeled data.

8 pages, 3 figures, accepted by IJCNN 2018

References in corpus (5)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Deep Learning Based Large-Scale Automatic Satellite Crosswalk Classification

Cited by in corpus (30)

• A Survey of Privacy Attacks in Machine Learning
• I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences
• Thieves on Sesame Street! Model Extraction of BERT-based APIs
• Stealing Neural Networks via Timing Side Channels
• Privacy-Preserving Brain-Computer Interfaces: A Systematic Review
• Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks
• DAWN: Dynamic Adversarial Watermarking of Neural Networks
• High Accuracy and High Fidelity Extraction of Neural Networks
• Entangled Watermarks as a Defense against Model Extraction
• A framework for the extraction of Deep Neural Networks by leveraging public data
• Dynamic Slicing for Deep Neural Networks
• The Threat of Adversarial Attacks on Machine Learning in Network Security -- A Survey
• Dataset Inference: Ownership Resolution in Machine Learning
• Morphence: Moving Target Defense Against Adversarial Examples
• Model extraction from counterfactual explanations
• Copycat CNN: Are Random Non-Labeled Data Enough to Steal Knowledge from Black-box Models?
• Towards Security Threats of Deep Learning Systems: A Survey
• Stealing Knowledge from Protected Deep Neural Networks Using Composite Unlabeled Data
• On the Effectiveness of Dataset Watermarking in Adversarial Settings
• Stateful Detection of Model Extraction Attacks
• Model Weight Theft With Just Noise Inputs: The Curious Case of the Petulant Attacker
• The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning
• Extraction of Complex DNN Models: Real Threat or Boogeyman?
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Proof-of-Learning: Definitions and Practice
• Stealing Deep Reinforcement Learning Models for Fun and Profit
• Careful What You Wish For: on the Extraction of Adversarially Trained Models
• DeepMimic: Mentor-Student Unlabeled Data Based Training
• Stealing the Invisible: Unveiling Pre-Trained CNN Models through Adversarial Examples and Timing Side-Channels
• Marich: A Query-efficient Distributionally Equivalent Model Extraction Attack using Public Data