ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detection

Abstract

The knowledge of a deep learning model may be transferred to a student model, leading to intellectual property infringement or vulnerability propagation. Detecting such knowledge reuse is nontrivial because the suspect models may not be white-box accessible and/or may serve different tasks. In this paper, we propose ModelDiff, a testing-based approach to deep learning model similarity comparison. Instead of directly comparing the weights, activations, or outputs of two models, we compare their behavioral patterns on the same set of test inputs. Specifically, the behavioral pattern of a model is represented as a decision distance vector (DDV), in which each element is the distance between the model's reactions to a pair of inputs. The knowledge similarity between two models is measured with the cosine similarity between their DDVs. To evaluate ModelDiff, we created a benchmark that contains 144 pairs of models that cover most popular model reuse methods, including transfer learning, model compression, and model stealing. Our method achieved 91.7% correctness on the benchmark, which demonstrates the effectiveness of using ModelDiff for model reuse detection. A study on mobile deep learning apps has shown the feasibility of ModelDiff on real-world models.

ISSTA 2021

References in corpus (24)

• Distilling the Knowledge in a Neural Network
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• DeepXplore: Automated Whitebox Testing of Deep Learning Systems
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Stealing Machine Learning Models via Prediction APIs
• Embedding Watermarks into Deep Neural Networks
• Pruning Filters for Efficient ConvNets
• DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
• Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detection
• SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
• Simple Black-box Adversarial Attacks
• Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring
• Insights on representational similarity in neural networks with canonical correlation
• Enhancing Adversarial Example Transferability with an Intermediate Level Attack
• Dynamic Slicing for Deep Neural Networks
• Testing DNN Image Classifiers for Confusion & Bias Errors
• Deep Neural Network Fingerprinting by Conferrable Adversarial Examples
• IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary
• A Survey of Binary Code Similarity
• A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning
• An Empirical Evaluation of Adversarial Robustness under Transfer Learning
• DeepSmartFuzzer: Reward Guided Test Generation For Deep Learning