EX-RAY: Distinguishing Injected Backdoor from Natural Features in Neural Networks by Examining Differential Feature Symmetry

Abstract

Backdoor attack injects malicious behavior to models such that inputs embedded with triggers are misclassified to a target label desired by the attacker. However, natural features may behave like triggers, causing misclassification once embedded. While they are inevitable, mis-recognizing them as injected triggers causes false warnings in backdoor scanning. A prominent challenge is hence to distinguish natural features and injected backdoors. We develop a novel symmetric feature differencing method that identifies a smallest set of features separating two classes. A backdoor is considered injected if the corresponding trigger consists of features different from the set of features distinguishing the victim and target classes. We evaluate the technique on thousands of models, including both clean and trojaned models, from the TrojAI rounds 2-4 competitions and a number of models on ImageNet. Existing backdoor scanning techniques may produce hundreds of false positives (i.e., clean models recognized as trojaned). Our technique removes 78-100% of the false positives (by a state-of-the-art scanner ABS) with a small increase of false negatives by 0-30%, achieving 17-41% overall accuracy improvement, and facilitates achieving top performance on the leaderboard. It also boosts performance of other scanners. It outperforms false positive removal methods using L2 distance and attribution techniques. We also demonstrate its potential in detecting a number of semantic backdoor attacks.

arXiv comment21 pages

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• Axiomatic Attribution for Deep Networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Blind Backdoors in Deep Learning Models
• Rethinking the Trigger of Backdoor Attack
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection
• Defending Neural Backdoors via Generative Distribution Modeling
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations
• On Certifying Robustness against Backdoor Attacks via Randomized Smoothing
• Weight Poisoning Attacks on Pre-trained Models
• Label-Consistent Backdoor Attacks
• Identifying and Controlling Important Neurons in Neural Machine Translation
• Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• One-pixel Signature: Characterizing CNN Models for Backdoor Detection
• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations