Adversarial Neuron Pruning Purifies Backdoored Deep Models
arXiv:2110.14430
Abstract
As deep neural networks (DNNs) are growing larger, their requirements for computational resources become huge, which makes outsourcing training more popular. Training in a third-party platform, however, may introduce potential risks that a malicious trainer will return backdoored DNNs, which behave normally on clean samples but output targeted misclassifications whenever a trigger appears at the test time. Without any knowledge of the trigger, it is difficult to distinguish or recover benign DNNs from backdoored ones. In this paper, we first identify an unexpected sensitivity of backdoored DNNs, that is, they are much easier to collapse and tend to predict the target label on clean samples when their neurons are adversarially perturbed. Based on these observations, we propose a novel model repairing method, termed Adversarial Neuron Pruning (ANP), which prunes some sensitive neurons to purify the injected backdoor. Experiments show, even with only an extremely small amount of clean data (e.g., 1%), ANP effectively removes the injected backdoor without causing obvious performance degradation.
arXiv commentTo appear in NeurIPS 2021
References in corpus (11)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Understanding Black-box Predictions via Influence Functions
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- Stronger Data Poisoning Attacks Break Data Sanitization Defenses
- Residual Convolutional CTC Networks for Automatic Speech Recognition
- Defending Neural Backdoors via Generative Distribution Modeling
- RAB: Provable Robustness Against Backdoor Attacks
- Label-Consistent Backdoor Attacks
- Anti-Backdoor Learning: Training Clean Models on Poisoned Data
- Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness
- Deep Partition Aggregation: Provable Defense against General Poisoning Attacks