Piracy Resistant Watermarks for Deep Neural Networks

Abstract

As companies continue to invest heavily in larger, more accurate and more robust deep learning models, they are exploring approaches to monetize their models while protecting their intellectual property. Model licensing is promising, but requires a robust tool for owners to claim ownership of models, i.e. a watermark. Unfortunately, current designs have not been able to address piracy attacks, where third parties falsely claim model ownership by embedding their own "pirate watermarks" into an already-watermarked model. We observe that resistance to piracy attacks is fundamentally at odds with the current use of incremental training to embed watermarks into models. In this work, we propose null embedding, a new way to build piracy-resistant watermarks into DNNs that can only take place at a model's initial training. A null embedding takes a bit string (watermark value) as input, and builds strong dependencies between the model's normal classification accuracy and the watermark. As a result, attackers cannot remove an embedded watermark via tuning or incremental training, and cannot add new pirate watermarks to already watermarked models. We empirically show that our proposed watermarks achieve piracy resistance and other watermark properties, over a wide range of tasks and models. Finally, we explore a number of adaptive counter-measures, and show our watermark remains robust against a variety of model modifications, including model fine-tuning, compression, and existing methods to detect/remove backdoors. Our watermarked models are also amenable to transfer learning without losing their watermark properties.

arXiv comment18 pages

References in corpus (8)

• Embedding Watermarks into Deep Neural Networks
• Robust Physical-World Attacks on Deep Learning Models
• Towards a Human-like Open-Domain Chatbot
• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks
• DAWN: Dynamic Adversarial Watermarking of Neural Networks
• Entangled Watermarks as a Defense against Model Extraction
• Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking
• [Extended version] Rethinking Deep Neural Network Ownership Verification: Embedding Passports to Defeat Ambiguity Attacks

Cited by in corpus (13)

• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• A Systematic Review on Model Watermarking for Neural Networks
• Intellectual Property Protection for Deep Learning Models: Taxonomy, Methods, Attacks, and Evaluations
• Identifying Appropriate Intellectual Property Protection Mechanisms for Machine Learning Models: A Systematization of Watermarking, Fingerprinting, Model Access, and Attacks
• RIGA: Covert and Robust White-Box Watermarking of Deep Neural Networks
• Deep Serial Number: Computational Watermarking for DNN Intellectual Property Protection
• Secure Watermark for Deep Neural Networks with Multi-task Learning
• Towards Practical Watermark for Deep Neural Networks in Federated Learning
• Proof-of-Learning: Definitions and Practice
• SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)
• Regulating Ownership Verification for Deep Neural Networks: Scenarios, Protocols, and Prospects
• WAFFLE: Watermarking in Federated Learning
• A survey of deep neural network watermarking techniques