EX-RAY: Distinguishing Injected Backdoor from Natural Features in Neural Networks by Examining Differential Feature Symmetry

Abstract

Backdoor attack injects malicious behavior to models such that inputs embedded with triggers are misclassified to a target label desired by the attacker. However, natural features may behave like triggers, causing misclassification once embedded. While they are inevitable, mis-recognizing them as injected triggers causes false warnings in backdoor scanning. A prominent challenge is hence to distinguish natural features and injected backdoors. We develop a novel symmetric feature differencing method that identifies a smallest set of features separating two classes. A backdoor is considered injected if the corresponding trigger consists of features different from the set of features distinguishing the victim and target classes. We evaluate the technique on thousands of models, including both clean and trojaned models, from the TrojAI rounds 2-4 competitions and a number of models on ImageNet. Existing backdoor scanning techniques may produce hundreds of false positives (i.e., clean models recognized as trojaned). Our technique removes 78-100% of the false positives (by a state-of-the-art scanner ABS) with a small increase of false negatives by 0-30%, achieving 17-41% overall accuracy improvement, and facilitates achieving top performance on the leaderboard. It also boosts performance of other scanners. It outperforms false positive removal methods using L2 distance and attribution techniques. We also demonstrate its potential in detecting a number of semantic backdoor attacks.

21 pages

References in corpus (12)

• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Defending Neural Backdoors via Generative Distribution Modeling
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations
• Weight Poisoning Attacks on Pre-trained Models
• Label-Consistent Backdoor Attacks
• Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• One-pixel Signature: Characterizing CNN Models for Backdoor Detection
• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations