Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

Abstract

Deep learning models have achieved high performance on many tasks, and thus have been applied to many security-critical scenarios. For example, deep learning-based face recognition systems have been used to authenticate users to access many security-sensitive applications like payment apps. Such usages of deep learning systems provide the adversaries with sufficient incentives to perform attacks against these systems for their adversarial purposes. In this work, we consider a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based authentication system, so that he can easily circumvent the system by leveraging the backdoor. Specifically, the adversary aims at creating backdoor instances, so that the victim learning system will be misled to classify the backdoor instances as a target label specified by the adversary. In particular, we study backdoor poisoning attacks, which achieve backdoor attacks using poisoning strategies. Different from all existing work, our studied poisoning strategies can apply under a very weak threat model: (1) the adversary has no knowledge of the model and the training set used by the victim system; (2) the attacker is allowed to inject only a small amount of poisoning samples; (3) the backdoor key is hard to notice even by human beings to achieve stealthiness. We conduct evaluation to demonstrate that a backdoor adversary can inject only around 50 poisoning samples, while achieving an attack success rate of above 90%. We are also the first work to show that a data poisoning attack can create physically implementable backdoors without touching the training process. Our work demonstrates that backdoor poisoning attacks pose real threats to a learning system, and thus highlights the importance of further investigation and proposing defense strategies against them.

References in corpus (10)

• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Understanding Black-box Predictions via Influence Functions
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Achieving Human Parity in Conversational Speech Recognition
• Is feature selection secure against training data poisoning?
• Generative Poisoning Attack Method Against Neural Networks
• Fingerprint Classification Based on Depth Neural Network
• Robust High Dimensional Sparse Regression and Matching Pursuit
• Robust High-Dimensional Linear Regression

Cited by in corpus (256)

• On the Opportunities and Risks of Foundation Models
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• How To Backdoor Federated Learning
• Analyzing Federated Learning through an Adversarial Lens
• Can You Really Backdoor Federated Learning?
• Spectral Signatures in Backdoor Attacks
• Mitigating Sybils in Federated Learning Poisoning
• A Survey on ChatGPT: AI-Generated Contents, Challenges, and Solutions
• Unleashing the potential of prompt engineering for large language models
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• Ditto: Fair and Robust Federated Learning Through Personalization
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• Input-Aware Dynamic Backdoor Attack
• Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• WaNet -- Imperceptible Warping-based Backdoor Attack
• Edge Intelligence: Architectures, Challenges, and Applications
• Poison Ink: Robust and Invisible Backdoor Attack
• Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
• Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
• Blind Backdoors in Deep Learning Models
• Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
• Rethinking the Trigger of Backdoor Attack
• MetaPoison: Practical General-purpose Clean-label Data Poisoning
• Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks
• Adversarial Attacks Against Deep Generative Models on Data: A Survey
• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks
• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• The Pitfalls of Simplicity Bias in Neural Networks
• Towards Adversarial Malware Detection: Lessons Learned from PDF-based Attacks
• AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
• Intellectual Property Protection for Deep Learning Models: Taxonomy, Methods, Attacks, and Evaluations
• Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks
• Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection
• You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
• Can Adversarial Weight Perturbations Inject Neural Backdoors?
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• On Certifying Robustness against Backdoor Attacks via Randomized Smoothing
• Security and Privacy Issues in Deep Learning
• DAWN: Dynamic Adversarial Watermarking of Neural Networks
• Explanation-Guided Backdoor Poisoning Attacks Against Malware Classifiers
• Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks
• Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey
• Weight Poisoning Attacks on Pre-trained Models
• RAB: Provable Robustness Against Backdoor Attacks
• Unlearnable Examples: Making Personal Data Unexploitable
• A Little Is Enough: Circumventing Defenses For Distributed Learning
• Backdoors in Neural Models of Source Code
• High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
• Label-Consistent Backdoor Attacks
• SAFELearning: Enable Backdoor Detectability In Federated Learning With Secure Aggregation
• Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
• Machine Learning Security in Industry: A Quantitative Survey
• Adversarial Attacks and Defenses on Graphs: A Review, A Tool and Empirical Studies
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• Adversarial Unlearning of Backdoors via Implicit Hypergradient
• Towards Probabilistic Verification of Machine Unlearning
• On Adversarial Bias and the Robustness of Fair Machine Learning
• Backdoor Attack in the Physical World
• FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
• Robust Backdoor Attacks against Deep Neural Networks in Real Physical World
• Radioactive data: tracing through training
• Adversarial Examples Make Strong Poisons
• On the Adversarial Robustness of LASSO Based Feature Selection
• A Master Key Backdoor for Universal Impersonation Attack against DNN-based Face Verification
• Machine Learning with Confidential Computing: A Systematization of Knowledge
• Training-free Lexical Backdoor Attacks on Language Models
• Adversarial Neuron Pruning Purifies Backdoored Deep Models
• What Do You See? Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors
• Universal Detection of Backdoor Attacks via Density-based Clustering and Centroids Analysis
• Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
• BadCM: Invisible Backdoor Attack Against Cross-Modal Learning
• Generalized Resilience and Robust Statistics
• DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
• Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking
• On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approach
• TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents
• Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
• ConFoc: Content-Focus Protection Against Trojan Attacks on Neural Networks
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• A Targeted Attack on Black-Box Neural Machine Translation with Parallel Data Poisoning
• Natural Backdoor Attack on Text Data
• Handcrafted Backdoors in Deep Neural Networks
• TrojanZoo: Towards Unified, Holistic, and Practical Evaluation of Neural Backdoors
• Backdoor Attacks for Remote Sensing Data with Wavelet Transform
• Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases
• Backdoor Attacks on Federated Meta-Learning
• Backdoor Attack through Frequency Domain
• A backdoor attack against LSTM-based text classification systems
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing
• Undistillable: Making A Nasty Teacher That CANNOT teach students
• Design of intentional backdoors in sequential models
• Turning Federated Learning Systems Into Covert Channels
• Poisoning and Backdooring Contrastive Learning
• Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
• On the Adversarial Robustness of Subspace Learning
• Hidden Backdoor Attack against Semantic Segmentation Models
• Online Data Poisoning Attack
• Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learning
• Removing Backdoor-Based Watermarks in Neural Networks with Limited Data
• Clean-Label Backdoor Attacks on Video Recognition Models
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations
• AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysis
• Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks
• Adversarial Filtering Based Evasion and Backdoor Attacks to EEG-Based Brain-Computer Interfaces
• Rethinking the Backdoor Attacks' Triggers: A Frequency Perspective
• Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks
• ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
• Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder
• HaS-Nets: A Heal and Select Mechanism to Defend DNNs Against Backdoor Attacks for Data Collection Scenarios
• Open-sourced Dataset Protection via Backdoor Watermarking
• Poisoned classifiers are not only backdoored, they are fundamentally broken
• Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection
• Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey
• Pitfalls in Machine Learning Research: Reexamining the Development Cycle
• RIGA: Covert and Robust White-Box Watermarking of Deep Neural Networks
• Light Can Hack Your Face! Black-box Backdoor Attack on Face Recognition Systems
• Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
• Deep Partition Aggregation: Provable Defense against General Poisoning Attacks
• Backdoor Attacks to Graph Neural Networks
• An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks
• SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics
• Embedding and Extraction of Knowledge in Tree Ensemble Classifiers
• Helen: Maliciously Secure Coopetitive Learning for Linear Models
• Robust and Differentially Private Mean Estimation
• TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation
• LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training
• SoK: Machine Learning Governance
• FooBaR: Fault Fooling Backdoor Attack on Neural Network Training
• M-to-N Backdoor Paradigm: A Multi-Trigger and Multi-Target Attack to Deep Learning Models
• Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers
• iBA: Backdoor Attack on 3D Point Cloud via Reconstructing Itself
• Backdoor Attacks on Crowd Counting
• Adversarial Machine Learning and Cybersecurity: Risks, Challenges, and Legal Implications
• How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data
• Poisoning MorphNet for Clean-Label Backdoor Attack to Point Clouds
• Poisoning Attacks to Local Differential Privacy Protocols for Key-Value Data
• BlockFLA: Accountable Federated Learning via Hybrid Blockchain Architecture
• ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks
• Defending Against Backdoor Attacks in Natural Language Generation
• Detect and remove watermark in deep neural networks via generative adversarial networks
• Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
• What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Space
• Subnet Replacement: Deployment-stage backdoor attack against deep neural networks in gray-box setting
• Dynamic Backdoors with Global Average Pooling
• Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
• Identifying Spurious Correlations for Robust Text Classification
• Repairing Deep Neural Networks: Fix Patterns and Challenges
• New Security Challenges on Machine Learning Inference Engine: Chip Cloning and Model Reverse Engineering
• FaceHack: Triggering backdoored facial recognition systems using facial characteristics
• Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks
• EX-RAY: Distinguishing Injected Backdoor from Natural Features in Neural Networks by Examining Differential Feature Symmetry
• Robust Training in High Dimensions via Block Coordinate Geometric Median Descent
• A Backdoor Approach with Inverted Labels Using Dirty Label-Flipping Attacks
• DBIA: Data-free Backdoor Injection Attack against Transformer Networks
• BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning
• Manipulating SGD with Data Ordering Attacks
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• GOAT: GPU Outsourcing of Deep Learning Training With Asynchronous Probabilistic Integrity Verification Inside Trusted Execution Environment
• Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption
• A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning
• Hu-Fu: Hardware and Software Collaborative Attack Framework against Neural Networks
• FedCom: A Byzantine-Robust Local Model Aggregation Rule Using Data Commitment for Federated Learning
• FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysis
• Influence Function based Data Poisoning Attacks to Top-N Recommender Systems
• Revealing Perceptible Backdoors, without the Training Set, via the Maximum Achievable Misclassification Fraction Statistic
• Mitigating Sybil Attacks on Differential Privacy based Federated Learning
• Benchmarking Spiking Neural Network Learning Methods with Varying Locality
• Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification
• PiDAn: A Coherence Optimization Approach for Backdoor Attack Detection and Mitigation in Deep Neural Networks
• A Backdoor Attack against 3D Point Cloud Classifiers
• Backdoor Attacks on Federated Learning with Lottery Ticket Hypothesis
• Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations
• Optimal Feature Manipulation Attacks Against Linear Regression
• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder
• Understanding the Limits of Unsupervised Domain Adaptation via Data Poisoning
• CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community
• Adversarial Parameter Defense by Multi-Step Risk Minimization
• Bait and Switch: Online Training Data Poisoning of Autonomous Driving Systems
• HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networks
• NTD: Non-Transferability Enabled Backdoor Detection
• Robust Federated Learning with Attack-Adaptive Aggregation
• Founding The Domain of AI Forensics
• Grounding Representation Similarity with Statistical Testing
• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
• A General Framework for Defending Against Backdoor Attacks via Influence Graph
• Two Sides of the Same Coin: Learning the Backdoor to Remove the Backdoor
• Machine Learning with Electronic Health Records is vulnerable to Backdoor Trigger Attacks
• Backdoor Attack on Hash-based Image Retrieval via Clean-label Data Poisoning
• DeepPoison: Feature Transfer Based Stealthy Poisoning Attack
• ONION: A Simple and Effective Defense Against Textual Backdoor Attacks
• Backdoor Learning Curves: Explaining Backdoor Poisoning Beyond Influence Functions
• DeepPayload: Black-box Backdoor Attack on Deep Learning Models through Neural Payload Injection
• Going Grayscale: The Road to Understanding and Improving Unlearnable Examples
• Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks
• PointBA: Towards Backdoor Attacks in 3D Point Cloud
• Enhancing the Antidote: Improved Pointwise Certifications against Poisoning Attacks
• Reaching Data Confidentiality and Model Accountability on the CalTrain
• Blackbox Trojanising of Deep Learning Models : Using non-intrusive network structure and binary alterations
• Selective Forgetting of Deep Networks at a Finer Level than Samples
• ABC-FL: Anomalous and Benign client Classification in Federated Learning
• Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
• Bias Busters: Robustifying DL-based Lithographic Hotspot Detectors Against Backdooring Attacks
• Hacking Neural Networks: A Short Introduction
• Towards Interpreting Recurrent Neural Networks through Probabilistic Abstraction
• Topological Detection of Trojaned Neural Networks
• Downscaling Attack and Defense: Turning What You See Back Into What You Get
• De-Pois: An Attack-Agnostic Defense against Data Poisoning Attacks
• 10 Security and Privacy Problems in Large Foundation Models
• A Framework of Randomized Selection Based Certified Defenses Against Data Poisoning Attacks
• An Evasion Attack against Stacked Capsule Autoencoder
• The Victim and The Beneficiary: Exploiting a Poisoned Model to Train a Clean Model on Poisoned Data
• ROSA: Finding Backdoors with Fuzzing
• Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks
• Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Space
• Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning
• Generative Models for Security: Attacks, Defenses, and Opportunities
• Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning
• Excess Capacity and Backdoor Poisoning
• Enhancing Model Robustness and Fairness with Causality: A Regularization Approach
• RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models
• Being Single Has Benefits. Instance Poisoning to Deceive Malware Classifiers
• Data Poisoning Attacks and Defenses to Crowdsourcing Systems
• Bounding The Number of Linear Regions in Local Area for Neural Networks with ReLU Activations
• SGBA: A Stealthy Scapegoat Backdoor Attack against Deep Neural Networks
• Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks
• Mental Models of Adversarial Machine Learning
• PoisHygiene: Detecting and Mitigating Poisoning Attacks in Neural Networks
• TRAPDOOR: Repurposing backdoors to detect dataset bias in machine learning-based genomic analysis
• Accumulative Poisoning Attacks on Real-time Data
• Extending Adversarial Attacks to Produce Adversarial Class Probability Distributions
• SpeechGuard: Online Defense against Backdoor Attacks on Speech Recognition Models
• Shape Defense Against Adversarial Attacks
• BadVFL: Backdoor Attacks in Vertical Federated Learning
• A Review of Computer Vision Methods in Network Security
• Protecting the integrity of the training procedure of neural networks
• Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning
• L-RED: Efficient Post-Training Detection of Imperceptible Backdoor Attacks without Access to the Training Set
• Law and Adversarial Machine Learning
• Broadly Applicable Targeted Data Sample Omission Attacks
• Reverse Engineering Imperceptible Backdoor Attacks on Deep Neural Networks for Detection and Training Set Cleansing
• A BIC-based Mixture Model Defense against Data Poisoning Attacks on Classifiers
• Shapley Homology: Topological Analysis of Sample Influence for Neural Networks
• Sequential Attacks on Kalman Filter-based Forward Collision Warning Systems
• On sensitivity of meta-learning to support data
• Semantic Host-free Trojan Attack
• Widen The Backdoor To Let More Attackers In