Backdoor Scanning for Deep Neural Networks through K-Arm Optimization

Abstract

Back-door attack poses a severe threat to deep learning systems. It injects hidden malicious behaviors to a model such that any input stamped with a special pattern can trigger such behaviors. Detecting back-door is hence of pressing need. Many existing defense techniques use optimization to generate the smallest input pattern that forces the model to misclassify a set of benign inputs injected with the pattern to a target label. However, the complexity is quadratic to the number of class labels such that they can hardly handle models with many classes. Inspired by Multi-Arm Bandit in Reinforcement Learning, we propose a K-Arm optimization method for backdoor detection. By iteratively and stochastically selecting the most promising labels for optimization with the guidance of an objective function, we substantially reduce the complexity, allowing to handle models with many classes. Moreover, by iteratively refining the selection of labels to optimize, it substantially mitigates the uncertainty in choosing the right labels, improving detection accuracy. At the time of submission, the evaluation of our method on over 4000 models in the IARPA TrojAI competition from round 1 to the latest round 4 achieves top performance on the leaderboard. Our technique also supersedes three state-of-the-art techniques in terms of accuracy and the scanning time needed.

References in corpus (19)

• Adam: A Method for Stochastic Optimization
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• How To Backdoor Federated Learning
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Input-Aware Dynamic Backdoor Attack
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks
• Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy
• Defending Neural Backdoors via Generative Distribution Modeling
• On Certifying Robustness against Backdoor Attacks via Randomized Smoothing
• Dynamic Backdoor Attacks Against Machine Learning Models
• Label-Consistent Backdoor Attacks
• Natural Backdoor Attack on Text Data
• Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• One-pixel Signature: Characterizing CNN Models for Backdoor Detection
• Deep Learning Backdoors
• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations

Cited by in corpus (6)

• Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
• TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation
• Trigger Hunting with a Topological Prior for Trojan Detection
• Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering
• TRIGS: Trojan Identification from Gradient-based Signatures
• AGNES: Abstraction-guided Framework for Deep Neural Networks Security