Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking

Abstract

The rise of machine learning as a service and model sharing platforms has raised the need of traitor-tracing the models and proof of authorship. Watermarking technique is the main component of existing methods for protecting copyright of models. In this paper, we show that distillation, a widely used transformation technique, is a quite effective attack to remove watermark embedded by existing algorithms. The fragility is due to the fact that distillation does not retain the watermark embedded in the model that is redundant and independent to the main learning task. We design ingrain in response to the destructive distillation. It regularizes a neural network with an ingrainer model, which contains the watermark, and forces the model to also represent the knowledge of the ingrainer. Our extensive evaluations show that ingrain is more robust to distillation attack and its robustness against other widely used transformation techniques is comparable to existing methods.

References in corpus (14)

• Distilling the Knowledge in a Neural Network
• Deep Learning with Differential Privacy
• ADADELTA: An Adaptive Learning Rate Method
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Poisoning Attacks against Support Vector Machines
• Embedding Watermarks into Deep Neural Networks
• Compressing Neural Networks with the Hashing Trick
• Deep Learning is Robust to Massive Label Noise
• Adversarial Frontier Stitching for Remote Neural Network Watermarking
• DeepSigns: A Generic Watermarking Framework for IP Protection of Deep Learning Models
• DeepMarks: A Digital Fingerprinting Framework for Deep Neural Networks
• Fraternal Twins: Unifying Attacks on Machine Learning and Digital Watermarking

Cited by in corpus (9)

• A Systematic Review on Model Watermarking for Neural Networks
• Intellectual Property Protection for Deep Learning Models: Taxonomy, Methods, Attacks, and Evaluations
• Identifying Appropriate Intellectual Property Protection Mechanisms for Machine Learning Models: A Systematization of Watermarking, Fingerprinting, Model Access, and Attacks
• Piracy Resistant Watermarks for Deep Neural Networks
• RIGA: Covert and Robust White-Box Watermarking of Deep Neural Networks
• Detect and remove watermark in deep neural networks via generative adversarial networks
• Deep Serial Number: Computational Watermarking for DNN Intellectual Property Protection
• Patch-wise++ Perturbation for Adversarial Targeted Attacks
• HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networks