Backdoor Pre-trained Models Can Transfer to All

Abstract

Pre-trained general-purpose language models have been a dominating component in enabling real-world natural language processing (NLP) applications. However, a pre-trained model with backdoor can be a severe threat to the applications. Most existing backdoor attacks in NLP are conducted in the fine-tuning phase by introducing malicious triggers in the targeted class, thus relying greatly on the prior knowledge of the fine-tuning task. In this paper, we propose a new approach to map the inputs containing triggers directly to a predefined output representation of the pre-trained NLP models, e.g., a predefined output representation for the classification token in BERT, instead of a target label. It can thus introduce backdoor to a wide range of downstream tasks without any prior knowledge. Additionally, in light of the unique properties of triggers in NLP, we propose two new metrics to measure the performance of backdoor attacks in terms of both effectiveness and stealthiness. Our experiments with various types of triggers show that our method is widely applicable to different fine-tuning tasks (classification and named entity recognition) and to different models (such as BERT, XLNet, BART), which poses a severe threat. Furthermore, by collaborating with the popular online model repository Hugging Face, the threat brought by our method has been confirmed. Finally, we analyze the factors that may affect the attack performance and share insights on the causes of the success of our backdoor attack.

References in corpus (15)

• Explaining and Harnessing Adversarial Examples
• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Pre-trained Models for Natural Language Processing: A Survey
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Pointer Sentinel Mixture Models
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN
• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
• STRIP: A Defence Against Trojan Attacks on Deep Neural Networks
• Weight Poisoning Attacks on Pre-trained Models
• RAB: Provable Robustness Against Backdoor Attacks

Cited by in corpus (11)

• Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
• Training-free Lexical Backdoor Attacks on Language Models
• Stealing the Decoding Algorithms of Language Models
• LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
• Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability
• PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning
• A Backdoor Approach with Inverted Labels Using Dirty Label-Flipping Attacks
• Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention
• PreGIP: Watermarking the Pretraining of Graph Neural Networks for Deep Intellectual Property Protection
• Moderator: Moderating Text-to-Image Diffusion Models through Fine-grained Context-based Policies
• Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks