STRIP: A Defence Against Trojan Attacks on Deep Neural Networks

Abstract

A recent trojan attack on deep neural network (DNN) models is one insidious variant of data poisoning attacks. Trojan attacks exploit an effective backdoor created in a DNN model by leveraging the difficulty in interpretability of the learned model to misclassify any inputs signed with the attacker's chosen trojan trigger. Since the trojan trigger is a secret guarded and exploited by the attacker, detecting such trojan inputs is a challenge, especially at run-time when models are in active operation. This work builds STRong Intentional Perturbation (STRIP) based run-time trojan attack detection system and focuses on vision system. We intentionally perturb the incoming input, for instance by superimposing various image patterns, and observe the randomness of predicted classes for perturbed inputs from a given deployed model---malicious or benign. A low entropy in predicted classes violates the input-dependence property of a benign model and implies the presence of a malicious input---a characteristic of a trojaned input. The high efficacy of our method is validated through case studies on three popular and contrasting datasets: MNIST, CIFAR10 and GTSRB. We achieve an overall false acceptance rate (FAR) of less than 1%, given a preset false rejection rate (FRR) of 1%, for different types of triggers. Using CIFAR10 and GTSRB, we have empirically achieved result of 0% for both FRR and FAR. We have also evaluated STRIP robustness against a number of trojan attack variants and adaptive attacks.

arXiv comment13 pages

References in corpus (12)

• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Convolutional Neural Networks for Sentence Classification
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• How To Backdoor Federated Learning
• Spectral Signatures in Backdoor Attacks
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• A Berkeley View of Systems Challenges for AI
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
• PoTrojan: powerful neural-level trojan designs in deep learning models
• BlackMarks: Blackbox Multibit Watermarking for Deep Neural Networks

Cited by in corpus (15)

• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Poison Ink: Robust and Invisible Backdoor Attack
• Backdoor Pre-trained Models Can Transfer to All
• Backdoor Scanning for Deep Neural Networks through K-Arm Optimization
• Piracy Resistant Watermarks for Deep Neural Networks
• Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data
• M-to-N Backdoor Paradigm: A Multi-Trigger and Multi-Target Attack to Deep Learning Models
• Backdoor Attacks on Crowd Counting
• Hijack Vertical Federated Learning Models As One Party
• New Security Challenges on Machine Learning Inference Engine: Chip Cloning and Model Reverse Engineering
• FaceHack: Triggering backdoored facial recognition systems using facial characteristics
• A Data Augmentation-based Defense Method Against Adversarial Attacks in Neural Networks
• Two Sides of the Same Coin: Learning the Backdoor to Remove the Backdoor