LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors

Abstract

Prompt-tuning has emerged as an attractive paradigm for deploying large-scale language models due to its strong downstream task performance and efficient multitask serving ability. Despite its wide adoption, we empirically show that prompt-tuning is vulnerable to downstream task-agnostic backdoors, which reside in the pretrained models and can affect arbitrary downstream tasks. The state-of-the-art backdoor detection approaches cannot defend against task-agnostic backdoors since they hardly converge in reversing the backdoor triggers. To address this issue, we propose LMSanitator, a novel approach for detecting and removing task-agnostic backdoors on Transformer models. Instead of directly inverting the triggers, LMSanitator aims to invert the predefined attack vectors (pretrained models' output when the input is embedded with triggers) of the task-agnostic backdoors, which achieves much better convergence performance and backdoor detection accuracy. LMSanitator further leverages prompt-tuning's property of freezing the pretrained model to perform accurate and fast output monitoring and input purging during the inference phase. Extensive experiments on multiple language models and NLP tasks illustrate the effectiveness of LMSanitator. For instance, LMSanitator achieves 92.8% backdoor detection accuracy on 960 models and decreases the attack success rate to less than 1% in most scenarios.

arXiv commentTo Appear in the Network and Distributed System Security (NDSS) Symposium 2024, 26 February - 1 March 2024, San Diego, CA, USA; typos corrected

References in corpus (35)

• Character-level Convolutional Networks for Text Classification
• NLTK: The Natural Language Toolkit
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Visualizing the Loss Landscape of Neural Nets
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• Spectral Signatures in Backdoor Attacks
• Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
• BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• ERNIE: Enhanced Language Representation with Informative Entities
• Making Pre-trained Language Models Better Few-shot Learners
• Analyzing Information Leakage of Updates to Natural Language Models
• Backdoor Pre-trained Models Can Transfer to All
• Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection
• Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy
• AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
• Defending Neural Backdoors via Generative Distribution Modeling
• Learn&Fuzz: Machine Learning for Input Fuzzing
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
• Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering
• BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models
• BadPrompt: Backdoor Attacks on Continuous Prompts
• Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks
• T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification
• LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
• Information Leakage in Embedding Models
• Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense

Cited by in corpus (1)

• LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors