Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Abstract

This work provides the community with a timely comprehensive review of backdoor attacks and countermeasures on deep learning. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdoor inspection, and post backdoor removal. Accordingly, we review countermeasures, and compare and analyze their advantages and disadvantages. We have also reviewed the flip side of backdoor attacks, which are explored for i) protecting intellectual property of deep learning models, ii) acting as a honeypot to catch adversarial example attacks, and iii) verifying data deletion requested by the data contributor.Overall, the research on defense is far behind the attack, and there is no single defense that can prevent all types of backdoor attacks. In some cases, an attacker can intelligently bypass existing defenses with an adaptive attack. Drawing the insights from the systematic review, we also present key areas for future research on the backdoor, such as empirical security evaluations from physical trigger attacks, and in particular, more efficient and practical countermeasures are solicited.

29 pages, 9 figures, 2 tables

References in corpus (63)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Federated Learning: Strategies for Improving Communication Efficiency
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Federated Learning for Mobile Keyboard Prediction
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Towards Federated Learning at Scale: System Design
• Binary Neural Networks: A Survey
• Can You Really Backdoor Federated Learning?
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Is feature selection secure against training data poisoning?
• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• Adversarial Examples: Opportunities and Challenges
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• A Berkeley View of Systems Challenges for AI
• Split learning for health: Distributed deep learning without sharing raw patient data
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Blind Backdoors in Deep Learning Models
• Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
• Rethinking the Trigger of Backdoor Attack
• MetaPoison: Practical General-purpose Clean-label Data Poisoning
• Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks
• Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy
• Defending Neural Backdoors via Generative Distribution Modeling
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations
• You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
• On Certifying Robustness against Backdoor Attacks via Randomized Smoothing
• DAWN: Dynamic Adversarial Watermarking of Neural Networks
• Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks
• Explanation-Guided Backdoor Poisoning Attacks Against Malware Classifiers
• PoTrojan: powerful neural-level trojan designs in deep learning models
• Weight Poisoning Attacks on Pre-trained Models
• RAB: Provable Robustness Against Backdoor Attacks
• Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures
• Entangled Watermarks as a Defense against Model Extraction
• Label-Consistent Backdoor Attacks
• Backdoors in Neural Models of Source Code
• Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization
• BlackMarks: Blackbox Multibit Watermarking for Deep Neural Networks
• End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things
• Biometric Backdoors: A Poisoning Attack Against Unsupervised Template Updating
• A Survey of Machine Learning Methods and Challenges for Windows Malware Classification
• Towards Probabilistic Verification of Machine Unlearning
• Backdoor attacks and defenses in feature-partitioned collaborative learning
• Piracy Resistant Watermarks for Deep Neural Networks
• ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks
• Natural Backdoor Attack on Text Data
• Defense Against Adversarial Attacks with Saak Transform
• The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Models
• Backdoor Attacks on Federated Meta-Learning
• Design of intentional backdoors in sequential models
• DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips
• Programmable Neural Network Trojan for Pre-Trained Feature Extractor
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks
• Adversarial Audio: A New Information Hiding Method and Backdoor for DNN-based Speech Recognition Models
• Universal Adversarial Attacks with Natural Triggers for Text Classification
• FaceHack: Triggering backdoored facial recognition systems using facial characteristics
• Revealing Perceptible Backdoors, without the Training Set, via the Maximum Achievable Misclassification Fraction Statistic
• Live Trojan Attacks on Deep Neural Networks
• AI Data poisoning attack: Manipulating game AI of Go

Cited by in corpus (19)

• Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
• Poison Ink: Robust and Invisible Backdoor Attack
• Rethinking the Trigger of Backdoor Attack
• Physical Adversarial Attack meets Computer Vision: A Decade Survey
• Backdoor Pre-trained Models Can Transfer to All
• Backdoor Attack in the Physical World
• Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
• Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models
• Synthetic Observational Health Data with GANs: from slow adoption to a boom in medical research and ultimately digital twins?
• TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation
• Poisoning MorphNet for Clean-Label Backdoor Attack to Point Clouds
• Hijack Vertical Federated Learning Models As One Party
• EX-RAY: Distinguishing Injected Backdoor from Natural Features in Neural Networks by Examining Differential Feature Symmetry
• Backdoor Attacks on Federated Learning with Lottery Ticket Hypothesis
• NTD: Non-Transferability Enabled Backdoor Detection
• Decamouflage: A Framework to Detect Image-Scaling Attacks on Convolutional Neural Networks
• Spinning Sequence-to-Sequence Models with Meta-Backdoors
• TRAPDOOR: Repurposing backdoors to detect dataset bias in machine learning-based genomic analysis
• Widen The Backdoor To Let More Attackers In