Spectral Signatures in Backdoor Attacks

Abstract

A recent line of work has uncovered a new form of data poisoning: so-called \emph{backdoor} attacks. These attacks are particularly dangerous because they do not affect a network's behavior on typical, benign data. Rather, the network only deviates from its expected output when triggered by a perturbation planted by an adversary. In this paper, we identify a new property of all known backdoor attacks, which we call \emph{spectral signatures}. This property allows us to utilize tools from robust statistics to thwart the attacks. We demonstrate the efficacy of these signatures in detecting and removing poisoned examples on real image sets and state of the art neural network architectures. We believe that understanding spectral signatures is a crucial first step towards designing ML systems secure against such backdoor attacks

arXiv comment16 pages, accepted to NIPS 2018

References in corpus (11)

• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Ensemble Adversarial Training: Attacks and Defenses
• Understanding Black-box Predictions via Influence Functions
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Poisoning Attacks against Support Vector Machines
• Robust Physical-World Attacks on Deep Learning Models
• Support Vector Machines under Adversarial Label Contamination
• Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring
• Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks
• Efficient Algorithms for Outlier-Robust Regression

Cited by in corpus (62)

• How To Backdoor Federated Learning
• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Input-Aware Dynamic Backdoor Attack
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Poison Ink: Robust and Invisible Backdoor Attack
• Blind Backdoors in Deep Learning Models
• Rethinking the Trigger of Backdoor Attack
• Adversarial Attacks Against Deep Generative Models on Data: A Survey
• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks
• Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection
• You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
• Can Adversarial Weight Perturbations Inject Neural Backdoors?
• Security and Privacy Issues in Deep Learning
• Explanation-Guided Backdoor Poisoning Attacks Against Malware Classifiers
• RAB: Provable Robustness Against Backdoor Attacks
• Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models
• A Little Is Enough: Circumventing Defenses For Distributed Learning
• Backdoors in Neural Models of Source Code
• Towards Probabilistic Verification of Machine Unlearning
• Adversarial Unlearning of Backdoors via Implicit Hypergradient
• Defending against Backdoor Attack on Deep Neural Networks
• Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
• DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
• Adversarial Learning in Statistical Classification: A Comprehensive Review of Defenses Against Attacks
• Backdoor Attacks on Federated Meta-Learning
• TrojanZoo: Towards Unified, Holistic, and Practical Evaluation of Neural Backdoors
• Handcrafted Backdoors in Deep Neural Networks
• Backdoor Attack through Frequency Domain
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing
• Min-Max Optimization without Gradients: Convergence and Applications to Adversarial ML
• NNoculation: Catching BadNets in the Wild
• Clean-Label Backdoor Attacks on Video Recognition Models
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations
• AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysis
• Rethinking the Backdoor Attacks' Triggers: A Frequency Perspective
• Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data
• Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection
• Backdoor Attacks to Graph Neural Networks
• LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
• Noise Sensitivity-Based Energy Efficient and Robust Adversary Detection in Neural Networks
• M-to-N Backdoor Paradigm: A Multi-Trigger and Multi-Target Attack to Deep Learning Models
• Backdoor Attacks on Crowd Counting
• Poisoning Deep Reinforcement Learning Agents with In-Distribution Triggers
• Poisoning MorphNet for Clean-Label Backdoor Attack to Point Clouds
• Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
• Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training
• A Backdoor Approach with Inverted Labels Using Dirty Label-Flipping Attacks
• Trigger Hunting with a Topological Prior for Trojan Detection
• Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification
• PiDAn: A Coherence Optimization Approach for Backdoor Attack Detection and Mitigation in Deep Neural Networks
• Revealing Perceptible Backdoors, without the Training Set, via the Maximum Achievable Misclassification Fraction Statistic
• A General Framework for Defending Against Backdoor Attacks via Influence Graph
• On Evaluating Neural Network Backdoor Defenses
• Backdoor Attack on Hash-based Image Retrieval via Clean-label Data Poisoning
• Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks
• Spinning Sequence-to-Sequence Models with Meta-Backdoors
• SGBA: A Stealthy Scapegoat Backdoor Attack against Deep Neural Networks
• Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks
• A Framework of Randomized Selection Based Certified Defenses Against Data Poisoning Attacks
• 10 Security and Privacy Problems in Large Foundation Models
• L-RED: Efficient Post-Training Detection of Imperceptible Backdoor Attacks without Access to the Training Set
• Reverse Engineering Imperceptible Backdoor Attacks on Deep Neural Networks for Detection and Training Set Cleansing