Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks

Abstract

Pre-trained models (PTMs) have been widely used in various downstream tasks. The parameters of PTMs are distributed on the Internet and may suffer backdoor attacks. In this work, we demonstrate the universal vulnerability of PTMs, where fine-tuned PTMs can be easily controlled by backdoor attacks in arbitrary downstream tasks. Specifically, attackers can add a simple pre-training task, which restricts the output representations of trigger instances to pre-defined vectors, namely neuron-level backdoor attack (NeuBA). If the backdoor functionality is not eliminated during fine-tuning, the triggers can make the fine-tuned model predict fixed labels by pre-defined vectors. In the experiments of both natural language processing (NLP) and computer vision (CV), we show that NeuBA absolutely controls the predictions for trigger instances without any knowledge of downstream tasks. Finally, we apply several defense methods to NeuBA and find that model pruning is a promising direction to resist NeuBA by excluding backdoored neurons. Our findings sound a red alarm for the wide use of PTMs. Our source code and models are available at \url{https://github.com/thunlp/NeuBA}.

arXiv commentPublished in Machine Intelligence Research (https://link.springer.com/article/10.1007/s11633-022-1377-5)

References in corpus (14)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Explaining and Harnessing Adversarial Examples
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
• Extracting Training Data from Large Language Models
• Word-level Textual Adversarial Attacking as Combinatorial Optimization
• BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• Backdoor Pre-trained Models Can Transfer to All
• A Tale of Evil Twins: Adversarial Inputs versus Poisoned Models
• Programmable Neural Network Trojan for Pre-Trained Feature Extractor
• A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning

Cited by in corpus (1)

• Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability