Post-breach Recovery: Protection against White-box Adversarial Examples for Leaked DNN Models

Abstract

Server breaches are an unfortunate reality on today's Internet. In the context of deep neural network (DNN) models, they are particularly harmful, because a leaked model gives an attacker "white-box" access to generate adversarial examples, a threat model that has no practical robust defenses. For practitioners who have invested years and millions into proprietary DNNs, e.g. medical imaging, this seems like an inevitable disaster looming on the horizon. In this paper, we consider the problem of post-breach recovery for DNN models. We propose Neo, a new system that creates new versions of leaked models, alongside an inference time filter that detects and removes adversarial examples generated on previously leaked models. The classification surfaces of different model versions are slightly offset (by introducing hidden distributions), and Neo detects the overfitting of attacks to the leaked model used in its generation. We show that across a variety of tasks and attack methods, Neo is able to filter out attacks from leaked models with very high accuracy, and provides strong protection (7--10 recoveries) against attackers who repeatedly breach the server. Neo performs well against a variety of strong adaptive attacks, dropping slightly in # of breaches recoverable, and demonstrates potential as a complement to DNN defenses in the wild.

References in corpus (11)

• ADADELTA: An Adaptive Learning Rate Method
• Theoretically Principled Trade-off between Robustness and Accuracy
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Fast is better than free: Revisiting adversarial training
• Defensive Distillation is Not Robust to Adversarial Examples
• Fixing Data Augmentation to Improve Adversarial Robustness
• Improving Adversarial Robustness of Ensembles with Diversity Training
• Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective
• Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks
• A Partial Break of the Honeypots Defense to Catch Adversarial Attacks
• Adversarial Attack across Datasets