Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks

Abstract

High-performance Deep Neural Networks (DNNs) are increasingly deployed in many real-world applications e.g., cloud prediction APIs. Recent advances in model functionality stealing attacks via black-box access (i.e., inputs in, predictions out) threaten the business model of such applications, which require a lot of time, money, and effort to develop. Existing defenses take a passive role against stealing attacks, such as by truncating predicted information. We find such passive defenses ineffective against DNN stealing attacks. In this paper, we propose the first defense which actively perturbs predictions targeted at poisoning the training objective of the attacker. We find our defense effective across a wide range of challenging datasets and DNN model stealing attacks, and additionally outperforms existing defenses. Our defense is the first that can withstand highly accurate model stealing attacks for tens of thousands of queries, amplifying the attacker's error rate up to a factor of 85$\times$ with minimal impact on the utility for benign users.

arXiv commentICLR 2020, Project page: https://resources.mpi-inf.mpg.de/d2/orekondy/predpoison/

References in corpus (7)

• Distilling the Knowledge in a Neural Network
• Deep Learning with Differential Privacy
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
• Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data
• A framework for the extraction of Deep Neural Networks by leveraging public data
• Exploring Connections Between Active Learning and Model Extraction

Cited by in corpus (21)

• A Systematic Review on Model Watermarking for Neural Networks
• DAWN: Dynamic Adversarial Watermarking of Neural Networks
• A Survey on Privacy for B5G/6G: New Privacy Challenges, and Research Directions
• Stealing Links from Graph Neural Networks
• Defend Data Poisoning Attacks on Voice Authentication
• Undistillable: Making A Nasty Teacher That CANNOT teach students
• Fusion: Efficient and Secure Inference Resilient to Malicious Servers
• Imitation Attacks and Defenses for Black-box Machine Translation Systems
• A Survey of AI-Related Cyber Security Risks and Countermeasures in Mobility-as-a-Service
• Optimal query complexity for private sequential learning against eavesdropping
• Towards Security Threats of Deep Learning Systems: A Survey
• MEGEX: Data-Free Model Extraction Attack against Gradient-Based Explainable AI
• On the Effectiveness of Dataset Watermarking in Adversarial Settings
• SoK: Machine Learning Governance
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Fingerprinting Multi-exit Deep Neural Network Models via Inference Time
• BODAME: Bilevel Optimization for Defense Against Model Extraction
• Simulating Unknown Target Models for Query-Efficient Black-box Attacks
• "What's in the box?!": Deflecting Adversarial Attacks by Randomly Deploying Adversarially-Disjoint Models
• 10 Security and Privacy Problems in Large Foundation Models
• Accelerating 2PC-based ML with Limited Trusted Hardware