Removing Backdoor-Based Watermarks in Neural Networks with Limited Data

Abstract

Deep neural networks have been widely applied and achieved great success in various fields. As training deep models usually consumes massive data and computational resources, trading the trained deep models is highly demanded and lucrative nowadays. Unfortunately, the naive trading schemes typically involves potential risks related to copyright and trustworthiness issues, e.g., a sold model can be illegally resold to others without further authorization to reap huge profits. To tackle this problem, various watermarking techniques are proposed to protect the model intellectual property, amongst which the backdoor-based watermarking is the most commonly-used one. However, the robustness of these watermarking approaches is not well evaluated under realistic settings, such as limited in-distribution data availability and agnostic of watermarking patterns. In this paper, we benchmark the robustness of watermarking, and propose a novel backdoor-based watermark removal framework using limited data, dubbed WILD. The proposed WILD removes the watermarks of deep models with only a small portion of training data, and the output model can perform the same as models trained from scratch without watermarks injected. In particular, a novel data augmentation method is utilized to mimic the behavior of watermark triggers. Combining with the distribution alignment between the normal and perturbed (e.g., occluded) data in the feature space, our approach generalizes well on all typical types of trigger contents. The experimental results demonstrate that our approach can effectively remove the watermarks without compromising the deep model performance for the original task with the limited access to training data.

References in corpus (9)

• Overcoming catastrophic forgetting in neural networks
• DeepPose: Human Pose Estimation via Deep Neural Networks
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Embedding Watermarks into Deep Neural Networks
• Adversarial Frontier Stitching for Remote Neural Network Watermarking
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Have You Stolen My Model? Evasion Attacks Against Deep Neural Network Watermarking Techniques

Cited by in corpus (8)

• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• Intellectual Property Protection for Deep Learning Models: Taxonomy, Methods, Attacks, and Evaluations
• RIGA: Covert and Robust White-Box Watermarking of Deep Neural Networks
• Detect and remove watermark in deep neural networks via generative adversarial networks
• Secure Watermark for Deep Neural Networks with Multi-task Learning
• Towards Practical Watermark for Deep Neural Networks in Federated Learning
• SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)
• Fine-tuning Is Not Enough: A Simple yet Effective Watermark Removal Attack for DNN Models